run · rs_tporoqk416ia1y19 · graded 2026-09-17 17:58:32graded on sitepublished · anyone who can see the task can open this
Share

jev-cascade-60

claude-haiku-4-5 + claude-opus-5 + jev-1.13.0via jev-cascade-60on ⚡ Jev as a router: cheaper than always-Opus?

Grader output

free-form JSON · this task's own · run here
0.895grader.score179 of 200 cases passed
score
0.895
passed
true
total cases
200
pass threshold
0
n_scored=200n_correct=179by_category={"law":0.87,"engineering":0.92}n_escalated=120n_timed_out=0n_unanswered=3cost_usd_total=2.3211latency_ms_p95=22582n_cost_metered=200n_cost_reported=200latency_ms_median=9120cost_unmetered_usd=0.005009n_escalation_broke=5n_escalation_fixed=39cases_errors_caught=0.9455cost_usd_per_correct=0.01299cases_accuracy_ci_low=0.86cases_escalation_rate=0.6n_escalation_reported=200cases_accuracy_ci_high=0.93n_first_answer_reported=180cases_escalation_precision=0.52

The task's own judge and grader ran on this site over the answers the agent submitted. This task's reference answers are not published; the site holds them and never sends them to a solver. Which model answered, what it cost and who typed remain the client's word.

Run frame

declared by the client · not validated
cases
200
duration total · client wall clock
2221.48s
cost · client-declared
$2.321
engine
claude-haiku-4-5 + claude-opus-5 + jev-1.13.0
answers
via jev-cascade-60

run configuration

declared by the client · not validated
agent
jev-cascade-60
launcher
tp 0.0.15.dev29+gab6189d
executor
tp 0.0.15.dev29+gab6189d
framework
jev-cascade-60
model · declared
claude-haiku-4-5 (trap.yaml) · claude-opus-5 (trap.yaml) · jev-1.13.0 (trap.yaml)
solution
https://github.com/Ruqii/trapstreet-solutions@41654d2b125d/model_routing/jev-cascade-60
task
https://github.com/trapstreet/trapstreet-tasks@7f36c68b831f/tasks/model_routing
trap
0.0.15.dev29+gab6189d

environment · self-reported, not validated

os
macOS 14.0
arch
arm64
kernel
Darwin 23.0.0
cpu
Apple M1 · 8 cores
memory
8.0 GB
python
3.13.7

skills & tools used

not reported

data coverage

identity
collected · tp · tp/0.0.15.dev29+gab6189d · 2026-09-17 17:58:30
model
collected · tp · tp/0.0.15.dev29+gab6189d · 2026-09-17 17:58:30
environment
collected · tp · tp/0.0.15.dev29+gab6189d · 2026-09-17 17:58:30
reproducibility
collected · tp · tp/0.0.15.dev29+gab6189d · 2026-09-17 17:58:30
skills
unsupported · tp runs a solver process; skills are a harness concept
tools
unsupported · tp does not observe the solver's tool calls
timing
collected · tp · tp/0.0.15.dev29+gab6189d · 2026-09-17 17:58:30
usage
collected · tp · tp/0.0.15.dev29+gab6189d · 2026-09-17 17:58:30 · $2.321 · reported
cases in this runcase_001: failcase_002: passcase_003: passcase_004: passcase_005: passcase_006: passcase_007: passcase_008: failcase_009: passcase_010: passcase_011: passcase_012: passcase_013: passcase_014: passcase_015: passcase_016: passcase_017: passcase_018: passcase_019: passcase_020: passcase_021: passcase_022: passcase_023: passcase_024: passcase_025: passcase_026: passcase_027: passcase_028: passcase_029: passcase_030: passcase_031: passcase_032: passcase_033: failcase_034: passcase_035: passcase_036: passcase_037: passcase_038: failcase_039: failcase_040: failcase_041: passcase_042: failcase_043: passcase_044: failcase_045: passcase_046: failcase_047: passcase_048: passcase_049: passcase_050: passcase_051: passcase_052: passcase_053: passcase_054: passcase_055: passcase_056: passcase_057: passcase_058: passcase_059: passcase_060: passcase_061: passcase_062: passcase_063: passcase_064: passcase_065: passcase_066: passcase_067: passcase_068: failcase_069: passcase_070: passcase_071: passcase_072: passcase_073: passcase_074: passcase_075: passcase_076: passcase_077: passcase_078: passcase_079: passcase_080: passcase_081: passcase_082: passcase_083: passcase_084: failcase_085: failcase_086: passcase_087: passcase_088: passcase_089: passcase_090: passcase_091: passcase_092: passcase_093: passcase_094: passcase_095: failcase_096: failcase_097: passcase_098: passcase_099: passcase_100: passcase_101: passcase_102: passcase_103: passcase_104: passcase_105: passcase_106: passcase_107: passcase_108: passcase_109: passcase_110: passcase_111: passcase_112: passcase_113: passcase_114: passcase_115: failcase_116: passcase_117: passcase_118: passcase_119: passcase_120: passcase_121: passcase_122: passcase_123: passcase_124: passcase_125: passcase_126: passcase_127: passcase_128: passcase_129: passcase_130: passcase_131: passcase_132: passcase_133: passcase_134: passcase_135: passcase_136: passcase_137: passcase_138: passcase_139: passcase_140: passcase_141: passcase_142: passcase_143: passcase_144: passcase_145: passcase_146: passcase_147: passcase_148: passcase_149: passcase_150: failcase_151: passcase_152: passcase_153: passcase_154: passcase_155: passcase_156: failcase_157: failcase_158: passcase_159: passcase_160: passcase_161: failcase_162: passcase_163: passcase_164: passcase_165: passcase_166: passcase_167: passcase_168: passcase_169: passcase_170: passcase_171: passcase_172: passcase_173: passcase_174: failcase_175: passcase_176: passcase_177: passcase_178: failcase_179: passcase_180: passcase_181: passcase_182: passcase_183: passcase_184: passcase_185: passcase_186: passcase_187: passcase_188: passcase_189: passcase_190: passcase_191: passcase_192: passcase_193: passcase_194: passcase_195: passcase_196: passcase_197: passcase_198: passcase_199: passcase_200: pass
per-case results · all 200, in case_id order
discuss this task →