all tasks / 😬😬 Does adding skills break the jobs an agent already did? / claude-haiku-4-5 / run_decanthcihg8vp7l

run · run_decanthcihg8vp7l · filed 2026-08-12 22:03:31

claude-haiku-4-5

claude-haiku-4-5-20251001Ruqii/trapstreet-solutions/core_capability_stacking_regression/claude-haiku-4-5-20251001@9ed11d1 ↗on 😬😬 Does adding skills break the jobs an agent already did?

Grader output

free-form JSON · this task's own

0.867grader.score62 of 108 cases passed

score: 0.867
passed: ✓
total cases: 108
passed cases: 62
skipped cases: 0
pass threshold: 0.500
cost total: $1.072
latency p95: 2.84 s
latency total: 286.00 s
latency median: 2.10 s

extras

arm_gap=0.1636by_cell=[object Object]n_scored=108primary_p=0.0293curve_shape=inflection@L1dose_by_cell=[object Object]primary_test=[object Object]bulk_probe_L4=[object Object]by_difficulty=[object Object]by_skill_pair=[object Object]primary_tiers=medium,hardby_stack_level=[object Object]completion_mean=0.9105by_overlap_class=[object Object]correctness_mean=0.8486curve_high_drops=0.1518,0.0297,0.0547n_solution_error=0by_failure_reason=[object Object]curve_low_overlap=[object Object]low_overlap_score=0.934n_skipped_no_gold=0by_competitor_dose=[object Object]curve_high_overlap=[object Object]high_overlap_score=0.7704secondary_sign_test_L3=[object Object]curve_high_by_difficulty=[object Object]n_failures_without_a_pair=42curve_low_overlap_all_tiers=[object Object]bled_by_instruction_strength=[object Object]curve_high_overlap_all_tiers=[object Object]solution_errors_by_stack_level=[object Object]score_excluding_solution_errors=0.8668

Fields here are the task author's — trap neither interprets nor validates them.

Run frame

written by trap · always present

cases: 108
duration total: 286.00s
cost: $1.072
engine: claude-haiku-4-5-20251001

environment · self-reported, not validated

os: macOS 14.0
arch: arm64
kernel: Darwin 23.0.0
cpu: Apple M1 · 8 cores
memory: 8.0 GB
trap: 0.0.14

per-case results · all 108, in case_id order

casesolutiondurationcostjudge

pair · s01/L0

pair · s01/L1

pair · s01/L2

pair · s01/L3

pair · s01/L4

pair · s02/L0

pair · s12/L0

pair · s12/L1

pair · s12/L2

pair · s12/L3

pair · s12/L4

pair · s02/L1

pair · s02/L2

pair · s02/L3

pair · s02/L4

pair · s03/L0

pair · s03/L1

pair · s03/L2

pair · s03/L3

pair · s03/L4

pair · s04/L0

pair · s04/L1

pair · s04/L2

pair · s04/L3

pair · s04/L4

pair · s05/L0

pair · s05/L1

pair · s05/L2

pair · s05/L3

pair · s05/L4

pair · s06/L0

pair · s06/L1

pair · s06/L2

pair · s06/L3

pair · s06/L4

pair · s07/L0

pair · s07/L1

pair · s07/L2

pair · s07/L3

pair · s07/L4

pair · s08/L0

pair · s08/L1

pair · s08/L2

pair · s08/L3

pair · s08/L4

pair · s09/L0

pair · s09/L1

pair · s09/L2

pair · s09/L3

pair · s09/L4

pair · s10/L0

pair · s10/L1

pair · s10/L2

pair · s10/L3

pair · s10/L4

pair · s11/L0

pair · s11/L1

pair · s11/L2

pair · s11/L3

pair · s11/L4

discuss this task →