run · run_decanthcihg8vp7l · filed 2026-08-12 22:03:31
Share

claude-haiku-4-5

Grader output

free-form JSON · this task's own
0.867grader.score62 of 108 cases passed
score
0.867
passed
total cases
108
passed cases
62
skipped cases
0
pass threshold
0.500
cost total
$1.072
latency p95
2.84 s
latency total
286.00 s
latency median
2.10 s

extras

arm_gap=0.1636by_cell={"L1/low":0.9556,"L2/low":0.9556,"L3/low":0.9278,"L4/low":0.8972,"L0/none":0.98…n_scored=108primary_p=0.0293curve_shape=inflection@L1dose_by_cell={"L1/high":[1,2,3,4],"L2/high":[3,4,5,6,7],"L3/high":[6,7,8,9,10,11],"L4/high":…primary_test={"note":"positive mean_diff means the high-overlap arm scored worse","mean_diff…bulk_probe_L4={"note":"L4 raises bulk in BOTH arms; registered separately so it cannot enter …by_difficulty={"easy":0.9111,"edge":0.7982,"hard":0.9003,"medium":0.8482}by_skill_pair={"calendar_create_event <- calendar_create_hold":3,"storage_copy_object <- stor…primary_tiers=["medium","hard"]by_stack_level={"L0":0.9833,"L1":0.8986,"L2":0.8597,"L3":0.8461,"L4":0.8045}completion_mean=0.9105by_overlap_class={"low":0.934,"high":0.7704,"none":0.9833}correctness_mean=0.8486curve_high_drops=[0.1518,0.0297,0.0547]n_solution_error=0by_failure_reason={"near_miss":14,"incomplete":6,"over_eager":4,"bad_arguments":3,"instruction_bl…curve_low_overlap={"L0":1,"L1":0.963,"L2":0.963,"L3":0.9259,"L4":0.9185}low_overlap_score=0.934n_skipped_no_gold=0by_competitor_dose={"1":1,"2":0.8667,"3":0.7111,"4":0.775,"5":0.6556,"6":0.9333,"7":0.72,"8":0.779…curve_high_overlap={"L0":1,"L1":0.8482,"L2":0.8185,"L3":0.7638,"L4":0.6934}high_overlap_score=0.7704secondary_sign_test_L3={"note":"one-sided exact sign test on discordant pairs from the pre-registered …curve_high_by_difficulty={"easy":{"L0":1,"L1":0.8,"L2":0.8,"L3":0.8,"L4":0.8},"edge":{"L0":0.9,"L1":0.83…n_failures_without_a_pair=42curve_low_overlap_all_tiers={"L0":0.9833,"L1":0.9556,"L2":0.9556,"L3":0.9278,"L4":0.8972}bled_by_instruction_strength={"blunt":22,"medium":11,"subtle":1}curve_high_overlap_all_tiers={"L0":0.9833,"L1":0.8417,"L2":0.7639,"L3":0.7645,"L4":0.7117}solution_errors_by_stack_level={}score_excluding_solution_errors=0.8668

Fields here are the task author's — trap neither interprets nor validates them.

Run frame

written by trap · always present
cases
108
duration total
286.00s
cost
$1.072
engine
claude-haiku-4-5-20251001

environment · self-reported, not validated

os
macOS 14.0
arch
arm64
kernel
Darwin 23.0.0
cpu
Apple M1 · 8 cores
memory
8.0 GB
trap
0.0.14
per-case results · all 108, in case_id order

pair · s01/L0

pair · s01/L1

pair · s01/L2

pair · s01/L3

pair · s01/L4

pair · s02/L0

pair · s12/L0

pair · s12/L1

pair · s12/L2

pair · s12/L3

pair · s12/L4

pair · s02/L1

pair · s02/L2

pair · s02/L3

pair · s02/L4

pair · s03/L0

pair · s03/L1

pair · s03/L2

pair · s03/L3

pair · s03/L4

pair · s04/L0

pair · s04/L1

pair · s04/L2

pair · s04/L3

pair · s04/L4

pair · s05/L0

pair · s05/L1

pair · s05/L2

pair · s05/L3

pair · s05/L4

pair · s06/L0

pair · s06/L1

pair · s06/L2

pair · s06/L3

pair · s06/L4

pair · s07/L0

pair · s07/L1

pair · s07/L2

pair · s07/L3

pair · s07/L4

pair · s08/L0

pair · s08/L1

pair · s08/L2

pair · s08/L3

pair · s08/L4

pair · s09/L0

pair · s09/L1

pair · s09/L2

pair · s09/L3

pair · s09/L4

pair · s10/L0

pair · s10/L1

pair · s10/L2

pair · s10/L3

pair · s10/L4

pair · s11/L0

pair · s11/L1

pair · s11/L2

pair · s11/L3

pair · s11/L4

discuss this task →