all tasks / Debugging in a vendor payout senario - not many cases, but hard 😬 / vendor-payout-multi-model-claude-sonnet-5 / run_5m0m3knp0g6orew2
run · run_5m0m3knp0g6orew2 · filed 2026-07-25 22:31:11
Share
vendor-payout-multi-model-claude-sonnet-5
Grader output
free-form JSON · this task's own0.750grader.score3 of 4 cases passed
- score
- 0.750
- passed
- ✓
- total cases
- 4
- passed cases
- 3
- skipped cases
- 0
- pass threshold
- 0.500
- cost total
- $0.305
- latency p95
- 139.48 s
- latency total
- 225.00 s
- latency median
- 30.86 s
score by category
supplier_change_dated
100%
royalty_change_with_b2b
100%
supplier_change_alltime
100%
srp_change_recalc_royalty
0%
extras
n_scored=4n_skipped_no_gold=0Fields here are the task author's — trap neither interprets nor validates them.
Run frame
written by trap · always present- cases
- 4
- duration total
- 225.00s
- cost
- $0.305
- engine
- claude-sonnet-5
environment · self-reported, not validated
- os
- macOS 14.0
- arch
- arm64
- kernel
- Darwin 23.0.0
- cpu
- Apple M1 · 8 cores
- memory
- 8.0 GB
- trap
- 0.0.8
per-case results · all 4, in case_id order
casesolutiondurationcostjudge