all tasks / 📊 🧐 Which pdf parser does the best job? read pdf chart / smolagents-split / run_x3axt2bt2xcf4w3o
run · run_x3axt2bt2xcf4w3o · filed 2026-08-26 12:23:07
Share
smolagents-split
claude-sonnet-5 + claude-opus-5Ruqii/trapstreet-solutions/pdf_chart_reasoning/smolagents-split@0a8c04e ↗on 📊 🧐 Which pdf parser does the best job? read pdf chart
Grader output
free-form JSON · this task's own0.739grader.score17 of 23 cases passed
- score
- 0.739
- passed
- ✗
- total cases
- 23
- passed cases
- 17
- skipped cases
- 0
- pass threshold
- 0.800
- cost total
- $2.172
- latency p95
- 158.06 s
- latency total
- 1203.00 s
- latency median
- 32.45 s
score by category
semantic
100%
abstention
100%
count_marks
100%
read_length
100%
cross_figure
100%
derived_value
25%
read_position
0%
extras
n_scored=23n_skipped_no_gold=0Fields here are the task author's — trap neither interprets nor validates them.
Run frame
written by trap · always present- cases
- 23
- duration total
- 1203.00s
- cost
- $2.172
- engine
- claude-sonnet-5 + claude-opus-5
environment · self-reported, not validated
- os
- macOS 14.0
- arch
- arm64
- kernel
- Darwin 23.0.0
- cpu
- Apple M1 · 8 cores
- memory
- 8.0 GB
- trap
- 0.0.14
per-case results · all 23, in case_id order
casesolutiondurationcostjudge