Which model and harness should answer questions over business data?
Multi-step questions over payment and business datasets.
Best in this use case · ranked only on systems that ran every core taskdeepseek-flash + claude-codemodel + harnessGot 84% of cases right, averaged over 1 core task.
Best value · at least 10% cheaper, still at least ¾ of the best scoreThe best is also the best valueNothing ranked here is at least 10% cheaper and still close to it.Cheapest that works: deepseek-flash + mini-loop · 48% · $0.14
bar = share of cases right · each row is one full system
💳 Bigger model or better harness? (DABStep 25)dabstepA data agent is handed one year of card payments for five merchants, the fee schedule that prices them, and the manual that explains how the schedule is read — and asked 25 questions about them. Five are put straight to the transactions: a count, a fraud rate. The other twenty ta25 cases · 8 runs · core taskOpen task: rules, cases, every run →
💳 Bigger model or better harness? (DABStep 25, same tools🔧)dabstep-shared-toolsA data agent is handed one year of card payments for five merchants, the fee schedule that prices them, and the manual that explains how the schedule is read — and asked 25 questions about them. Five are put straight to the transactions: a count, a fraud rate. The other twenty ta25 cases · 6 runs · extra task: not in the averageOpen task: rules, cases, every run →