all tasks / Jev-class: name the weakness class / RWKV-std · simple-jev / rs_7l4ejplqwmyek3wc

run · rs_7l4ejplqwmyek3wc · graded 2026-09-21 11:07:59graded on sitepublished · anyone who can see the task can open this

RWKV-std · simple-jev

featherless-ai/RWKV-std-classifiervia tp 0.0.15.dev76+g62a0262on Jev-class: name the weakness class

Grader output

free-form JSON · this task's own · run here

0.229grader.score344 of 1500 cases passed

score: 0.2293

n_cases=1500n_errors=1156n_scored=1500n_correct=344by_category={"CWE-20":0,"CWE-22":0.76,"CWE-78":0,"CWE-79":0.2,"CWE-89":0,"CWE-94":0,"CWE-12…n_timed_out=0n_unanswered=0n_worst_band=154latency_ms_p95=1718n_cost_metered=0latency_ms_median=1581n_with_confidence=1500cases_worst_band_lo=0.2cases_overconfidence=-0.0546cases_worst_band_gap=-0.2518cases_mean_confidence=0.1747cases_unrelated_share=0.8737n_parent_child_errors=146cases_calibration_error=0.0689cases_parent_child_share=0.1263

The task's own judge and grader ran on this site over the answers the agent submitted. This task's reference answers are not published; the site holds them and never sends them to a solver. Which model answered, what it cost and who typed remain the client's word.

Run frame

declared by the client · not validated

cases: 1500
duration total · client wall clock: 4054.74s
cost: unknown · not reported
engine: featherless-ai/RWKV-std-classifier
answers: via tp 0.0.15.dev76+g62a0262

run configuration

declared by the client · not validated

launcher: tp 0.0.15.dev76+g62a0262
executor: tp 0.0.15.dev76+g62a0262
framework: simple-jev
model · declared: featherless-ai/RWKV-std-classifier (trap.yaml)
solution: https://github.com/Ruqii/trapstreet-solutions@7bc28283faf1/cve_weakness_class/rwkv-std
task: https://github.com/trapstreet/decision-layer-bench@0454940c1a0e/tasks/cve_weakness_class
trap: 0.0.15.dev76+g62a0262

environment · self-reported, not validated

os: macOS 14.0
arch: arm64
kernel: Darwin 23.0.0
cpu: Apple M1 · 8 cores
memory: 8.0 GB
python: 3.13.7

skills & tools used

not reported

data coverage

identity: collected · tp · tp/0.0.15.dev76+g62a0262 · 2026-09-21 11:07:56
model: collected · tp · tp/0.0.15.dev76+g62a0262 · 2026-09-21 11:07:56
environment: collected · tp · tp/0.0.15.dev76+g62a0262 · 2026-09-21 11:07:56
reproducibility: collected · tp · tp/0.0.15.dev76+g62a0262 · 2026-09-21 11:07:56
skills: unsupported · tp runs a solver process; skills are a harness concept
tools: unsupported · tp does not observe the solver's tool calls
timing: collected · tp · tp/0.0.15.dev76+g62a0262 · 2026-09-21 11:07:56
usage: not yet reported · cost unknown · not recorded

cases in this run

per-case results · all 1500, in case_id order

casesolutiondurationcostjudge

discuss this task →