run · run-mpizw3io-3
claude-pdf
status
scored · passed
total score
0.895
cases
17 passed · 2 failed
latency
69.17s
cost
$3.684
tokens
—
scored
2026-05-23 23:44:03
duration
69s
Run summary
- score
- 0.895
- passed
- ✓
- total cases
- 19
- passed cases
- 17
- skipped cases
- 0
- pass threshold
- 0.800
- tokens total
- null
- cost total
- $3.684
- latency p95
- 7.25 s
- latency total
- 69.17 s
- latency median
- 3.24 s
score by category
dates
100%
money
86%
clauses
88%
deposit
100%
scenario
100%
scenario_reasoning
100%
extras
n_scored=19n_skipped_no_gold=0Solution metadata
Self-reported by the solution. Not validated.
- repo
- https://github.com/Ruqii/trapstreet-solutions
- engine
- claude-opus-4-7
Per-case results
| case | exit | duration | metrics |
|---|---|---|---|
| break_clause | 0 | 2921 ms | id=break_clausetype=booleanmodel=claude-opus-4-7score=1category=clausesusd_cost=0.119616difficulty=mediumagent_answer=noinput_tokens=64output_tokens=5expected_answer=nomatcher_results=[object Object]cache_read_input_tokens=78854cache_creation_input_tokens=0 |
| deposit_amount | 0 | 2576 ms | id=deposit_amounttype=numericmodel=claude-opus-4-7score=1category=moneyusd_cost=0.119451difficulty=easyagent_answer=£2,250.00input_tokens=28output_tokens=10expected_answer=2250matcher_results=[object Object]cache_read_input_tokens=78854cache_creation_input_tokens=0 |
| deposit_dispute_escalation | 0 | 4695 ms | id=deposit_dispute_escalationtype=textmodel=claude-opus-4-7score=1category=depositusd_cost=0.123786difficulty=hardagent_answer=The dispute will be submitted to the ICE (Independent Case Examiner of The Dispute Service) for adjudication, and all parties agree to co-operate with the adjudication.input_tokens=62output_tokens=61expected_answer=The dispute may be submitted to the Independent Case Examiner (ICE) for adjudication.matcher_results=[object Object],[object Object]cache_read_input_tokens=78854cache_creation_input_tokens=0 |
| deposit_scheme | 0 | 3211 ms | id=deposit_schemetype=textmodel=claude-opus-4-7score=1category=clausesusd_cost=0.120801difficulty=mediumagent_answer=The Dispute Service (Tenancy Deposit Scheme).input_tokens=38output_tokens=26expected_answer=TDSmatcher_results=[object Object],[object Object]cache_read_input_tokens=78854cache_creation_input_tokens=0 |
| early_surrender | 0 | 3008 ms | id=early_surrendertype=booleanmodel=claude-opus-4-7score=0category=clausesusd_cost=0.119481difficulty=hardagent_answer=Noinput_tokens=55output_tokens=5expected_answer=yesmatcher_results=[object Object]cache_read_input_tokens=78854cache_creation_input_tokens=0 |
| early_surrender_economic_incentive | 0 | 4206 ms | id=early_surrender_economic_incentivetype=booleanmodel=claude-opus-4-7score=1category=scenario_reasoningusd_cost=0.126381difficulty=expertagent_answer=No. The clause only specifies that the tenant is liable for "any difference in rental payments... to the extent that the replacement Tenant will be paying a lower amount." It does not provide for any refund or credit if the replacement tenant pays a higher rent.input_tokens=125output_tokens=83expected_answer=nomatcher_results=[object Object],[object Object],[object Object]cache_read_input_tokens=78854cache_creation_input_tokens=0 |
| fixed_term_departure_notice | 0 | 2810 ms | id=fixed_term_departure_noticetype=booleanmodel=claude-opus-4-7score=1category=clausesusd_cost=0.119571difficulty=hardagent_answer=Yesinput_tokens=51output_tokens=7expected_answer=yesmatcher_results=[object Object]cache_read_input_tokens=78854cache_creation_input_tokens=0 |
| governing_act | 0 | 3665 ms | id=governing_acttype=textmodel=claude-opus-4-7score=1category=clausesusd_cost=0.120351difficulty=hardagent_answer=Housing Act 1988input_tokens=78output_tokens=12expected_answer=Housing Act 1988matcher_results=[object Object],[object Object]cache_read_input_tokens=78854cache_creation_input_tokens=0 |
| inventory_referenced | 0 | 2519 ms | id=inventory_referencedtype=booleanmodel=claude-opus-4-7score=1category=clausesusd_cost=0.119676difficulty=hardagent_answer=yesinput_tokens=63output_tokens=6expected_answer=yesmatcher_results=[object Object]cache_read_input_tokens=78854cache_creation_input_tokens=0 |
| late_rent_interest_rate | 0 | 3659 ms | id=late_rent_interest_ratetype=textmodel=claude-opus-4-7score=1category=moneyusd_cost=0.120831difficulty=hardagent_answer=3% per annum above the Bank of England base rateinput_tokens=70output_tokens=20expected_answer=3% per annum above Bank of England base ratematcher_results=[object Object],[object Object],[object Object]cache_read_input_tokens=78854cache_creation_input_tokens=0 |
| pets_allowed | 0 | 2888 ms | id=pets_allowedtype=textmodel=claude-opus-4-7score=1category=clausesusd_cost=0.119706difficulty=mediumagent_answer=with landlord consentinput_tokens=45output_tokens=10expected_answer=with landlord consentmatcher_results=[object Object],[object Object],[object Object]cache_read_input_tokens=78854cache_creation_input_tokens=0 |
| post_fixed_term_extension | 0 | 2934 ms | id=post_fixed_term_extensiontype=textmodel=claude-opus-4-7score=1category=clausesusd_cost=0.121986difficulty=hardagent_answer=The agreement automatically extends for a further period of six months from the expiration of the term, with rent increasing by 5%.input_tokens=47output_tokens=40expected_answer=the tenancy automatically extends for six monthsmatcher_results=[object Object],[object Object]cache_read_input_tokens=78854cache_creation_input_tokens=0 |
| rent_increase_scope | 0 | 3730 ms | id=rent_increase_scopetype=textmodel=claude-opus-4-7score=1category=moneyusd_cost=0.119871difficulty=hardagent_answer=The automatic extension period.input_tokens=46output_tokens=12expected_answer=the automatic extension period onlymatcher_results=[object Object],[object Object],[object Object]cache_read_input_tokens=78854cache_creation_input_tokens=0 |
| rent_payment_day | 0 | 2649 ms | id=rent_payment_daytype=textmodel=claude-opus-4-7score=0category=moneyusd_cost=0.119226difficulty=mediumagent_answer=5thinput_tokens=28output_tokens=7expected_answer=on or prior 5th of the monthmatcher_results=[object Object],[object Object],[object Object]cache_read_input_tokens=78854cache_creation_input_tokens=0 |
| rent_year2 | 0 | 5755 ms | id=rent_year2type=numericmodel=claude-opus-4-7score=1category=moneyusd_cost=1.479907difficulty=mediumagent_answer=£2100.00input_tokens=48output_tokens=9expected_answer=2100matcher_results=[object Object]cache_read_input_tokens=0cache_creation_input_tokens=78854 |
| rent_year3 | 0 | 3881 ms | id=rent_year3type=numericmodel=claude-opus-4-7score=1category=moneyusd_cost=0.120051difficulty=mediumagent_answer=£2400.00input_tokens=73output_tokens=9expected_answer=2400matcher_results=[object Object]cache_read_input_tokens=78854cache_creation_input_tokens=0 |
| scenario_leave_22mo_replacement_1mo_gap | 0 | 7247 ms | id=scenario_leave_22mo_replacement_1mo_gaptype=numericmodel=claude-opus-4-7score=1category=scenariousd_cost=0.150801difficulty=hardagent_answer=Months remaining at surrender: 36 − 22 = 14 months early. Replacement found after 1 month, so replacement occupies remaining 13 months. Rent at month 22 of a 36-month term starting 05/09/2022: - Months 1–12 (05/09/2022–04/09/2023): £1,950 - Months 13–24 (05/09/2023–04/09/2024): £2,100 - Months 25–36 (05/09/2024–04/09/2025): £2,400 Month 22 falls in the £2,100 period. The 1-month gap is at month 23 = £2,100. The replacement's 13 months: months 24 (£2,100) + months 25–36 (12 × £2,400 = £28,800) = £30,900. (a) Rent gap: £2,100 (b) Letting fee: £30,900 × 13.2% = £4,078.80 (c) Inventory: £144 × 14 / 36 = £56.00 (d) Admin: £480 × 14 / 36 = £186.67 Total: £2,100 + £4,078.80 + £56.00 + £186.67 = **£6,421.47**input_tokens=273output_tokens=379expected_answer=£6,421.47matcher_results=[object Object]cache_read_input_tokens=78854cache_creation_input_tokens=0 |
| term_start | 0 | 3572 ms | id=term_starttype=datemodel=claude-opus-4-7score=1category=datesusd_cost=0.119691difficulty=easyagent_answer=05/09/2022input_tokens=39output_tokens=11expected_answer=05/09/2022matcher_results=[object Object]cache_read_input_tokens=78854cache_creation_input_tokens=0 |
| total_rent_fixed_term | 0 | 3239 ms | id=total_rent_fixed_termtype=numericmodel=claude-opus-4-7score=1category=moneyusd_cost=0.123321difficulty=hardagent_answer=12 × £1950 + 12 × £2100 + 12 × £2400 = £23,400 + £25,200 + £28,800 = £77,400.input_tokens=66output_tokens=54expected_answer=77400matcher_results=[object Object]cache_read_input_tokens=78854cache_creation_input_tokens=0 |