all tasks / 💳 Bigger model or better harness? (DABStep 25) / kimi-k3 · claude-code / rs_z2y04t4vxekpaxq5

run · rs_z2y04t4vxekpaxq5 · graded 2026-09-13 01:41:16graded on sitepublished · anyone who can see the task can open this

kimi-k3 · claude-code

kimi-k3via claude-codeon 💳 Bigger model or better harness? (DABStep 25)

Grader output

free-form JSON · this task's own · run here

0.640grader.score16 of 25 cases passed

score: 0.64
passed: false
total cases: 25
passed cases: 16
pass threshold: 0.8

n_wrong=4n_scored=25by_category={"easy":0.8,"hard":0.6}n_unanswered=5cost_usd_total=14.4364latency_ms_p95=1700317n_easy_correct=4n_hard_correct=12latency_ms_total=12329949cost_cases_priced=25latency_ms_median=124977n_skipped_no_gold=0cost_unanswered_usd=8.2967cost_per_correct_usd=0.9023

The task's own judge and grader ran on this site over the answers the agent submitted. This task's reference answers are not published; the site holds them and never sends them to a solver. Which model answered, what it cost and who typed remain the client's word.

Run frame

declared by the client · not validated

cases: 25
duration total · client wall clock: 12372.75s
cost · client-declared: $14.436
engine: kimi-k3
answers: via claude-code

run configuration

declared by the client · not validated

agent: claude-code
launcher: tp 0.0.15.dev29+gab6189d
executor: tp 0.0.15.dev29+gab6189d
framework: claude-code
model · declared: kimi-k3 (trap.yaml)
model config: effort=max (Kimi's Claude Code guide) · harness=Claude Code 2.1.260, headless, as shipped (not --bare) · thinking=adaptive · isolation=sandbox-exec, dabstep/sandbox.py @ 1b08d42 · model_slots=kimi-k3[1m]; haiku slot kimi-k2.7-code · time_limit_s=1700 · auto_compact_window=1000000 · web_search_and_fetch=disallowed
solution: https://github.com/Ruqii/trapstreet-solutions@1b08d42c8e7f/dabstep/claude-code-kimi-k3
task: https://github.com/trapstreet/trapstreet-tasks@e7e12e652aa2/tasks/dabstep
trap: 0.0.15.dev29+gab6189d

environment · self-reported, not validated

os: macOS 14.0
arch: arm64
kernel: Darwin 23.0.0
cpu: Apple M1 · 8 cores
memory: 8.0 GB
python: 3.13.7

skills & tools used

skills used: not reported
skills installed / available: not reported / not reported
tools: Bash ×275 · Write ×130 · Read ×67 · Grep ×3 · Glob ×3 · Edit ×1 · TaskOutput ×1

data coverage

identity: collected · tp · tp/0.0.15.dev29+gab6189d · 2026-09-13 01:40:41
model: collected · dabstep-solutions · trapstreet-solutions dabstep (trap.yaml, transcripts) · 2026-09-13 23:08:57
environment: collected · tp · tp/0.0.15.dev29+gab6189d · 2026-09-13 01:40:41
reproducibility: collected · tp · tp/0.0.15.dev29+gab6189d · 2026-09-13 01:40:41
skills: unsupported · tp runs a solver process; skills are a harness concept
tools: collected · dabstep-solutions · trapstreet-solutions dabstep (trap.yaml, transcripts) · 2026-09-13 23:08:57
timing: collected · tp · tp/0.0.15.dev29+gab6189d · 2026-09-13 01:40:41
usage: collected · tp · tp/0.0.15.dev29+gab6189d · 2026-09-13 01:40:41 · $14.436 · reported

cases in this run

per-case results · all 25, in case_id order

casesolutiondurationcostjudge

discuss this task →