run · rs_dvs1co6a8zvluv0f · graded 2026-11-09 08:39:36graded on sitepublished · anyone who can see the task can open this
Share

mistralai/mistral-small-24b-instruct-2501 / collections v0.2

mistralai/mistral-small-24b-instruct-2501via openrouter-fresh-case v0.2on US Collections Decisions v0.2

Grader output

free-form JSON · this task's own · run here
0.232grader.score0 of 72 cases passed
score
0.23179287918871253
passed
false
total cases
72
cases_passed=0cases_measured=72cases_with_critical_failure=54

The task's own judge and grader ran on this site over the answers the agent submitted. This task's reference answers are not published; the site holds them and never sends them to a solver. Which model answered, what it cost and who typed remain the client's word.

Run frame

declared by the client · not validated
cases
72
duration total · case execution sum
347.26s · 72/72 cases
cost · client-declared
$0.010
engine
mistralai/mistral-small-24b-instruct-2501
answers
via openrouter-fresh-case v0.2

run configuration

declared by the client · not validated
agent
openrouter-fresh-case v0.2
model · declared
mistralai/mistral-small-24b-instruct-2501 (openrouter-api-import)
model config
schema=json_schema_strict · external_tools=false · actual_providers=DeepInfra · completion_token_limit=24576 · fresh_context_per_case=true
task
https://github.com/trapstreet/trapstreet-tasks-private@bc34b1c0cf91/tasks/us_collections_decisions_v02
evaluation
ev_b7aae04e1a6438b7
lockfiles
frozen-plan.json 98267f34bdf6

environment · self-reported, not validated

os
darwin
arch
arm64

skills & tools used

not reported

data coverage

identity
collected · openrouter-api-import · 2026-11-07 06:54:16
model
collected · openrouter-api-import · 2026-11-07 06:54:16
environment
collected · openrouter-api-import · 2026-11-07 06:54:16
reproducibility
collected · openrouter-api-import · 2026-11-07 06:54:16
skills
not yet reported
tools
not yet reported
timing
not yet reported
usage
not yet reported · cost unknown · not recorded
cases in this runcd_0836b925b443: failcd_0892bfc96db2: failcd_0aa717badcaf: partialcd_11d39d765dbc: partialcd_1599d4132eb8: failcd_1a5e7c403cd0: failcd_1ad3c8adc9da: failcd_1c0a1b2e9bee: failcd_1df7249b3769: partialcd_1e353f458f51: partialcd_228a50d83a91: partialcd_28a003c800a7: partialcd_2992917d83ca: partialcd_2be0d5c85051: partialcd_31576687e54d: failcd_371319ec040f: failcd_38850ed8ee6a: partialcd_39a24e2ee72e: failcd_3da4f098b2e3: partialcd_469bcd2ba134: partialcd_57de46de2061: failcd_61d9b0881f19: partialcd_633228c6852b: partialcd_633fad4665a7: partialcd_634515a3462a: failcd_6a0dd1f5457b: failcd_6e1a76c91545: partialcd_70cc8ae14538: partialcd_73214caf4caf: partialcd_76a5e70fc871: partialcd_7761d0659364: partialcd_7a94c4b6ad86: failcd_7b4c97548db3: partialcd_8075127ea902: failcd_8220cd59c9d8: partialcd_850b813103ff: partialcd_8aae940a97f6: failcd_8c10a5e83a3a: failcd_9154865ef40d: partialcd_986f7c115d52: failcd_9951439b5338: partialcd_99ca11dc61d2: failcd_9c985112763e: failcd_9d75fa96d342: partialcd_9e93e24082f0: partialcd_9fee56590dda: failcd_a3188b6d6622: failcd_a687ead489c4: partialcd_a7be6234f36d: partialcd_b0e39bcdfdd4: failcd_b4ce85ab97ef: partialcd_b5b3d2bc3fd7: partialcd_b83554ba7a71: partialcd_c23260b8dc5f: partialcd_c5178392eeaf: failcd_c7149460dc89: failcd_c868adbe4589: partialcd_c87f1151cf30: failcd_cbf929091177: partialcd_d5995bee2ef5: failcd_d67f84e93ee2: partialcd_d953aab50d9e: partialcd_df1c3b0922c2: failcd_dfd3074a7403: failcd_e016f95bb269: failcd_e33ee4408365: failcd_e73a04fef7f0: failcd_ec0d33491e9c: failcd_ee55c373d8da: partialcd_f016b8294135: failcd_f29bfb8023f1: partialcd_f36788f52d19: partial
per-case results · all 72, in case_id order
discuss this task →