all tasks / Can your solution read my legal document correctly? / pdf-reader-smolagents-split / run_nkm5h1nhc0kkky77
run · run_nkm5h1nhc0kkky77 · filed 2026-08-06 21:59:10
Share
pdf-reader-smolagents-split
claude-opus-5 + claude-sonnet-5Ruqii/trapstreet-solutions/pdf_reader/smolagents-split@8eeeb65 ↗on Can your solution read my legal document correctly?
Grader output
free-form JSON · this task's own0.947grader.score18 of 19 cases passed
- score
- 0.947
- passed
- ✓
- total cases
- 19
- passed cases
- 18
- skipped cases
- 0
- pass threshold
- 0.800
- cost total
- $0.497
- latency p95
- 212.61 s
- latency total
- 604.00 s
- latency median
- 17.47 s
score by category
dates
100%
money
86%
clauses
100%
deposit
100%
scenario
100%
scenario_reasoning
100%
extras
n_scored=19n_skipped_no_gold=0Fields here are the task author's — trap neither interprets nor validates them.
Run frame
written by trap · always present- cases
- 19
- duration total
- 604.00s
- cost
- $0.497
- engine
- claude-opus-5 + claude-sonnet-5
environment · self-reported, not validated
- os
- macOS 14.0
- arch
- arm64
- kernel
- Darwin 23.0.0
- cpu
- Apple M1 · 8 cores
- memory
- 8.0 GB
- trap
- 0.0.14 · older than 0.0.15.dev23, the build the site's graded-run protocol expects
per-case results · all 19, in case_id order
casesolutiondurationcostjudge