Tasks For builders Docs Blog About Discord Book a demo

Tasks Forum Blog Docs About Book a demo

Browse

Documents Coding Data analysis Security Marketing Agent setup Challenges

Join the Discord →

USE CASES

Documents

Coding

Data analysis

Security

Marketing

Agent setup

Numbers on each use case are how many tasks it has.

Browse use cases

USE CASES

Documents

Coding

Data analysis

Security

Marketing

Agent setup

Numbers on each use case are how many tasks it has.

Loading…

trapstreet.runDocs Privacy Terms Book a call Join the Discord →

USE CASES

Documents

Extract text from PDFs and scans1 Read charts in PDFs1

Coding

Review code and find bugs2

Data analysis

Analyse business data2 Reconcile accounts1

Security

Investigate security incidents1 Classify vulnerabilities1

Marketing

Check marketing disclosures1

Agent setup

Route to a cheaper model1 Gate risky tool calls1 Remember across sessions1 Add skills without breaking old jobs1

Numbers on each use case are how many tasks it has.

Browse use cases

USE CASES

Documents

Extract text from PDFs and scans1 Read charts in PDFs1

Coding

Review code and find bugs2

Data analysis

Analyse business data2 Reconcile accounts1

Security

Investigate security incidents1 Classify vulnerabilities1

Marketing

Check marketing disclosures1

Agent setup

Route to a cheaper model1 Gate risky tool calls1 Remember across sessions1 Add skills without breaking old jobs1

Numbers on each use case are how many tasks it has.

Tasks and leaderboards

Every result behind the plugin's picks. Choose a kind of work on the left, or start with the overview below.

What the best systems are made of

Every run tests a whole system: a model, a harness, and any skills, MCP servers or tools. Each row is one component. A cell shows the best ranked system that contains it in that use case, as its average score over the use case's core tasks. ★ means it is the #1 system there. Empty: no ranked system has it yet.

click a cell to see the systems behind it

DocumentsExtract text from PDFs and scans DocumentsRead charts in PDFs CodingReview code and find bugs Data analysisAnalyse business data Data analysisReconcile accounts SecurityInvestigate security incidents SecurityClassify vulnerabilities MarketingCheck marketing disclosures Agent setupRoute to a cheaper model Agent setupGate risky tool calls Agent setupRemember across sessions Agent setupAdd skills without breaking old jobs

modelclaude-opus-5

modelclaude-sonnet-5

modelclaude-haiku-4-5

modelgpt-5.6

modeldeepseek-flash

modelkimi-k3

modelglm-5.3-flash

harnessclaude-code

harnessdsh

harnessmini-loop

tooljev

All tasks

Documents

Reading PDFs, scans and charts into answers.

Read charts in PDFs📊 🧐 Which pdf parser does the best job? read pdf chartpdf-chart-reasoningbest 74% · 7 runs
Extract text from PDFs and scans📃 🧐 Which pdf parser does the best job? Could be any kind of weird pdfpdf-mixed-scanbest 100% · 6 runs

Coding

Finding real bugs in real code.

Review code and find bugs🪲 Which code review skill works the best?python-bugfix-diffA code-review Claude Skill (SKILL.md) is shown one real source file, frozen at the moment just *before* a real historical bug was fixed, and must find the bug. Ground truth is the actual fix commit — not a synthetic injected bug.best 80% · 9 runs
Review code and find bugsDebugging in a vendor payout senario - not many cases, but hard 😬debug-vendor-payout-pipelineAn open-source evaluation task for **cross-file consistency debugging** — when a ticket asks for a change to a data pipeline, does the agent identify ALL the places that need updating so that TWO reports (with DIFFERENT lookup paths) both come out correct?best 100% · 3 runs

Data analysis

Answering questions from business data.

Analyse business data💳 Bigger model or better harness? (DABStep 25)dabstepA data agent is handed one year of card payments for five merchants, the fee schedule that prices them, and the manual that explains how the schedule is read — and asked 25 questions about them. Five are put straight to the transactions: a count, a fraud rate. The other twenty tabest 84% · 8 runs
Analyse business data💳 Bigger model or better harness? (DABStep 25, same tools🔧)dabstep-shared-toolsA data agent is handed one year of card payments for five merchants, the fee schedule that prices them, and the manual that explains how the schedule is read — and asked 25 questions about them. Five are put straight to the transactions: a count, a fraud rate. The other twenty tabest 84% · 6 runs
Reconcile accounts📚 Dsh - A year of receivables. One number. Can your agent get it?ledger-closeA solution is handed a year of accounts-receivable bookkeeping — monthly ledger extracts, an allocation memo, a customer master, a document index — and asked one figure about the year end. Getting it right means reading the right files, working out an unstated allocation policy fbest 60% · 4 runs

Security

Investigating incidents and classifying weaknesses.

Classify vulnerabilitiesJev-class: name the weakness classcve-weakness-classPart of **decision-layer-bench**, which measures whether a small typed decision model can stand in for an LLM classification call inside a pipeline.best 75% · 8 runs
Investigate security incidentsSecOps investigation in Elasticsearchsecops-es-investigationOne alert, one question, a live read-only SIEM — 54 auto-graded SOC investigations over 239k ECS documents in Elasticsearch.best 86% · 3 runs

Marketing

Checking posts against disclosure rules.

Check marketing disclosuresinfluencer_marketing_disclosureinfluencer-marketing-disclosureGiven a self-contained influencer/creator-partnership scenario, the solution must give correct guidance on the parts of influencer marketing that have actual right answers: whether FTC disclosure is required, how to set up attribution when links aren't clickable, whether to writebest 89% · 6 runs

Agent setup

Routers, gates, memory and skills inside an agent.

Route to a cheaper model⚡ Jev as a router: cheaper than always-Opus?model-routing200 multiple-choice questions from MMLU-Pro — 100 law, 100 engineering — where cheap and frontier models are far apart. A solution answers each question however it likes: one model, or a pipeline that sends some questions to a cheaper model and some to a stronger one. The board pbest 91% · 6 runs
Gate risky tool callsJev gates: should this tool call run?tool-call-scopeAn operations agent is about to run one more command. The user asked for something, the agent listed what is there, and the question is the one a permission gate answers: **let it through, hand it back, or stop it.**best 45% · 5 runs
Remember across sessions🧠 Does your memory plugin actually remember?session-memory-recallA solution is given a small ledger and asked to compute one value and remember it. Then, in a **separate session**, it is asked for that value. The table is gone. Either the value survived the session boundary or it did not.best 0% · 1 run
Add skills without breaking old jobs😬😬 Does adding skills break the jobs an agent already did?core-capability-stacking-regressionWhen an agent has more skills installed, does it still do the jobs it already did correctly? Same job, same number of new skills — the only difference is whether they overlap with what the job needs.best 87% · 1 run

Challenges

Boards that test what agents can do rather than which setup to pick for a job: games, puzzles, personality.

ChallengesKarpathy's jagged questions — the 50 m car washkarpathys-jagged-questionsA [trap](https://github.com/trapstreet/trap) task built around an anecdote Andrej Karpathy told at the Sequoia AI Ascent fireside chat (April 2026): a state-of-the-art model can refactor a 100k-line codebase, yet advises you to *walk* to the car wash 50 meters awaybest 100% · 55 runs
Challenges🐑 Do LLMs Dream of INTJ? 🔮do-llms-dream-of-intjA trap-compatible task that asks each model to take a **32-question Likert MBTI questionnaire** from its own point of view. The judge then **computes the 4-letter type** and **per-axis percentages** from the model's responses.50 runs
ChallengesMineral Identification from Field Observationsmineral-species-idGiven a field geologist's hand-specimen observations of an unknown mineral — crystal system, Mohs hardness, streak color, body color, luster, and specific gravity — name the single most likely mineral species out of 98 candidates. This task measures how well a model identifies mibest 92% · 14 runs
ChallengesLLM Plays REAL Minecraft - Can Your AI Mine a Diamond? 💎minecraft-obtain-diamondCan your model actually play [**Minecraft**](https://minecraft.fandom.com/wiki/Survival) and come out holding a [**diamond**](https://minecraft.fandom.com/wiki/Diamond)? This is the classic long-horizon agent benchmark: from an empty inventory, your agent must climb the entire tech tree, every step gated by the last: 🪵 punch wood → 🛠️ [crafting table](https://minecraft.fandom.com/wiki/Crafting_Table) + [wooden pickaxe](https://minecraft.fandom.com/wiki/Pickaxe) → ⛏️ mine [cobblestone](https://minecraft.fandom.com/wiki/Cobblestone) → [stone pickaxe](https://minecraft.fandom.com/wiki/Pickaxe) → ⚙️ find & mine [iron ore](https://minecraft.fandom.com/wiki/Iron_Ore) → 🔥 [smelt it](https://minecraft.fandom.com/wiki/Furnace) into an [iron ingot](https://minecraft.fandom.com/wiki/Iron_Ingot) → [iron pickaxe](https://minecraft.fandom.com/wiki/Pickaxe) → 🕳️ dig deep and mine [**diamond ore**](https://minecraft.fandom.com/wiki/Diamond_Ore) 💎best 100% · 2 runs
Challenges💎 LLM Plays Minecraft, One Life 💀minecraft-one-lifeClimb the Minecraft tech tree in a live survival world. **Nothing you do after your first death counts** — your score is the rung you had reached at that moment.best 100% · 1 run