run · run_x0ah1zo1jwdnps8v
docling-claude
Ruqii/trapstreet-solutions/docling-claude@80cb002 ↗
total score
0.632
engine
docling + claude-opus-4-7
cases
12 passed · 7 failed
latency
147.05s
cost
$1.695
submitted
2026-05-23 23:55:10
Run summary
- score
- 0.632
- passed
- ✗
- total cases
- 19
- passed cases
- 12
- skipped cases
- 0
- pass threshold
- 0.800
- tokens total
- null
- cost total
- $1.695
- latency p95
- 13.20 s
- latency total
- 147.05 s
- latency median
- 7.12 s
score by category
dates
0%
money
29%
clauses
100%
deposit
100%
scenario
0%
scenario_reasoning
100%
extras
n_scored=19n_skipped_no_gold=0Per-case results
12 pass7 fail
answer
The deposit amount is not legible in the extracted text.
expected
2250
- id
- deposit_amount
- type
- numeric
- model
- claude-opus-4-7
- category
- money
- usd_cost
- $0.086
- difficulty
- easy
- input_tokens
- 30
- output_tokens
- 21
- matcher_results
- [object Object]
- cache_read_input_tokens
- 56,211
- cache_creation_input_tokens
- 0
answer
The 1st of the month.
expected
on or prior 5th of the month
- id
- rent_payment_day
- type
- text
- model
- claude-opus-4-7
- category
- money
- usd_cost
- $0.086
- difficulty
- medium
- input_tokens
- 30
- output_tokens
- 12
- matcher_results
- [object Object],[object Object],[object Object]
- cache_read_input_tokens
- 56,211
- cache_creation_input_tokens
- 0
answer
The document states rent increases but the actual figures appear as garbled/encoded text in the extraction and cannot be read as a clear GBP amount.
expected
2100
- id
- rent_year2
- type
- numeric
- model
- claude-opus-4-7
- category
- money
- usd_cost
- $0.089
- difficulty
- medium
- input_tokens
- 50
- output_tokens
- 49
- matcher_results
- [object Object]
- cache_read_input_tokens
- 56,211
- cache_creation_input_tokens
- 0
answer
N/A
expected
2400
- id
- rent_year3
- type
- numeric
- model
- claude-opus-4-7
- category
- money
- usd_cost
- $0.086
- difficulty
- medium
- input_tokens
- 75
- output_tokens
- 7
- matcher_results
- [object Object]
- cache_read_input_tokens
- 56,211
- cache_creation_input_tokens
- 0
answer
Months remaining at surrender: 36 − 22 = 14 months. Gap = 1 month, so replacement covers 13 months.
Rent schedule per the agreement:
- Months 1–12: original rent (call it R1)
- Months 13–24: increased rent R2
- Months 25–36: further increased rent R3
The tenant is 22 months in, so months 23–36 remain (14 months).
- Month 23–24: at R2 (the second-tier rent)
- Months 25–36: at R3 (the third-tier rent)
The document's rent figures (clause 8a/8b) are not legible in the extract, so I'll express the answer in terms of the rents:
(a) Gap rent (1 month, month 23) = R2
(b) Replacement covers 13 months: month 24 at R2, months 25–36 (12 months) at R3.
Letting fee = 13.2% × (1×R2 + 12×R3)
(c) Inventory check-in = £144 × 14/36 = £56.00
(d) Administration = £480 × 14/36 = £186.67
**Total = R2 + 0.132 × (R2 + 12·R3) + £242.67**
A specific GBP total cannot be computed because the rent amounts in clauses 8(a) and 8(b) are not readable in the provided markdown.
expected
£6,421.47
- id
- scenario_leave_22mo_replacement_1mo_gap
- type
- numeric
- model
- claude-opus-4-7
- category
- scenario
- usd_cost
- $0.121
- difficulty
- hard
- input_tokens
- 275
- output_tokens
- 435
- matcher_results
- [object Object]
- cache_read_input_tokens
- 56,211
- cache_creation_input_tokens
- 0
answer
16/09/2019
expected
05/09/2022
- id
- term_start
- type
- date
- model
- claude-opus-4-7
- category
- dates
- usd_cost
- $0.086
- difficulty
- easy
- input_tokens
- 41
- output_tokens
- 11
- matcher_results
- [object Object]
- cache_read_input_tokens
- 56,211
- cache_creation_input_tokens
- 0
answer
Based on the document, the rent figures (£ amounts) in clauses 8(a) and 8(b) are not legible in the OCR output — the actual numeric values for the monthly rent and the increased rents are not present in the extracted markdown, so the total cannot be computed.
expected
77400
- id
- total_rent_fixed_term
- type
- numeric
- model
- claude-opus-4-7
- category
- money
- usd_cost
- $0.092
- difficulty
- hard
- input_tokens
- 68
- output_tokens
- 90
- matcher_results
- [object Object]
- cache_read_input_tokens
- 56,211
- cache_creation_input_tokens
- 0