all tasks / 💳 Bigger model or better harness? (DABStep 25, same tools🔧) / deepseek-flash · pi / rs_dx5o8tzf7i95a163

run · rs_dx5o8tzf7i95a163 · graded 2026-09-20 00:42:21graded on sitepublished · anyone who can see the task can open this

deepseek-flash · pi

deepseek-flashvia pion 💳 Bigger model or better harness? (DABStep 25, same tools🔧)

Grader output

free-form JSON · this task's own · run here

0.840grader.score21 of 25 cases passed

score: 0.84
passed: true
total cases: 25
passed cases: 21
pass threshold: 0.8

n_wrong=4n_scored=25by_category={"easy":1,"hard":0.8}n_unanswered=0cost_usd_total=0.8081latency_ms_max=1376100latency_ms_p95=1038669n_easy_correct=5n_hard_correct=16n_at_time_limit=0latency_ms_total=8135597cost_cases_priced=25latency_ms_median=149063n_skipped_no_gold=0cost_unanswered_usd=0cost_per_correct_usd=0.0385

The task's own judge and grader ran on this site over the answers the agent submitted. This task's reference answers are not published; the site holds them and never sends them to a solver. Which model answered, what it cost and who typed remain the client's word.

Run frame

declared by the client · not validated

cases: 25
duration total · client wall clock: 8177.04s
cost · client-declared: $0.808
engine: deepseek-flash
answers: via pi

run configuration

declared by the client · not validated

agent: pi
launcher: tp 0.0.15.dev29+gab6189d
executor: tp 0.0.15.dev29+gab6189d
framework: pi
model · declared: deepseek-flash (trap.yaml)
model config: tools=the board's three shared tools only (same-tools/tools.py through a pi extension; Pi has no MCP), verified by same-tools/ · harness=Pi 0.85.1, headless (-p), --no-builtin-tools · thinking=on, level high · isolation=sandbox-exec, dabstep/sandbox.py @ 5e50dc5 · time_limit_s=1700 · other_model_calls=case_019 only: the model used the shared python tool to call the cost proxy with the API key from its environment, liste · web_search_and_fetch=none (Pi ships no web tools)
solution: https://github.com/Ruqii/trapstreet-solutions@53936cba6602/dabstep/same-tools/pi-deepseek-flash
task: https://github.com/trapstreet/trapstreet-tasks@07993beb95ed/tasks/dabstep
trap: 0.0.15.dev29+gab6189d

environment · self-reported, not validated

os: macOS 14.0
arch: arm64
kernel: Darwin 23.0.0
cpu: Apple M1 · 8 cores
memory: 8.0 GB
python: 3.13.7

skills & tools used

skills used: not reported
skills installed / available: not reported / not reported
tools: mcp__bench__run_python ×666 · mcp__bench__read_file ×163 · mcp__bench__list_dir ×30

data coverage

identity: collected · tp · tp/0.0.15.dev29+gab6189d · 2026-09-20 00:42:19
model: collected · dabstep-solutions · trapstreet-solutions dabstep/same-tools (trap.yaml, transcri · 2026-09-20 14:46:50
environment: collected · tp · tp/0.0.15.dev29+gab6189d · 2026-09-20 00:42:19
reproducibility: collected · tp · tp/0.0.15.dev29+gab6189d · 2026-09-20 00:42:19
skills: unsupported · tp runs a solver process; skills are a harness concept
tools: collected · dabstep-solutions · trapstreet-solutions dabstep/same-tools (trap.yaml, transcri · 2026-09-20 14:46:50
timing: collected · tp · tp/0.0.15.dev29+gab6189d · 2026-09-20 00:42:19
usage: collected · tp · tp/0.0.15.dev29+gab6189d · 2026-09-20 00:42:19 · $0.808 · reported

cases in this run

per-case results · all 25, in case_id order

casesolutiondurationcostjudge

discuss this task →