all tasks / Jev-class: name the weakness class / RWKV-mid · simple-jev / rs_20wabu4r1rnfy1dl

run · rs_20wabu4r1rnfy1dl · graded 2026-09-21 14:10:37graded on sitepublished · anyone who can see the task can open this

RWKV-mid · simple-jev

featherless-ai/RWKV-mid-classifiervia tp 0.0.15.dev76+g62a0262on Jev-class: name the weakness class

Grader output

free-form JSON · this task's own · run here

0.118grader.score177 of 1500 cases passed

score: 0.118

n_cases=1500n_errors=1323n_scored=1500n_correct=177by_category={"CWE-20":0,"CWE-22":0,"CWE-78":0,"CWE-79":0,"CWE-89":0,"CWE-94":0,"CWE-120":0,…n_timed_out=0n_unanswered=0n_worst_band=847latency_ms_p95=1496n_cost_metered=0latency_ms_median=1387n_with_confidence=1500cases_worst_band_lo=0.1cases_overconfidence=0.0963cases_worst_band_gap=0.1662cases_mean_confidence=0.2143cases_unrelated_share=0.7604n_parent_child_errors=317cases_calibration_error=0.149cases_parent_child_share=0.2396

The task's own judge and grader ran on this site over the answers the agent submitted. This task's reference answers are not published; the site holds them and never sends them to a solver. Which model answered, what it cost and who typed remain the client's word.

Run frame

declared by the client · not validated

cases: 1500
duration total · client wall clock: 3397.68s
cost: unknown · not reported
engine: featherless-ai/RWKV-mid-classifier
answers: via tp 0.0.15.dev76+g62a0262

run configuration

declared by the client · not validated

launcher: tp 0.0.15.dev76+g62a0262
executor: tp 0.0.15.dev76+g62a0262
framework: simple-jev
model · declared: featherless-ai/RWKV-mid-classifier (trap.yaml)
solution: https://github.com/Ruqii/trapstreet-solutions@572ee4fd21a2/cve_weakness_class/rwkv-mid
task: https://github.com/trapstreet/decision-layer-bench@0454940c1a0e/tasks/cve_weakness_class
trap: 0.0.15.dev76+g62a0262

environment · self-reported, not validated

os: macOS 14.0
arch: arm64
kernel: Darwin 23.0.0
cpu: Apple M1 · 8 cores
memory: 8.0 GB
python: 3.13.7

skills & tools used

not reported

data coverage

identity: collected · tp · tp/0.0.15.dev76+g62a0262 · 2026-09-21 14:10:34
model: collected · tp · tp/0.0.15.dev76+g62a0262 · 2026-09-21 14:10:34
environment: collected · tp · tp/0.0.15.dev76+g62a0262 · 2026-09-21 14:10:34
reproducibility: collected · tp · tp/0.0.15.dev76+g62a0262 · 2026-09-21 14:10:34
skills: unsupported · tp runs a solver process; skills are a harness concept
tools: unsupported · tp does not observe the solver's tool calls
timing: collected · tp · tp/0.0.15.dev76+g62a0262 · 2026-09-21 14:10:34
usage: not yet reported · cost unknown · not recorded

cases in this run

per-case results · all 1500, in case_id order

casesolutiondurationcostjudge

discuss this task →