An open-source evaluation task for PDF OCR / vision-based document reading. Useful as a basic sanity check when building AI agents that ingest PDFs — invoice processors, legal-doc reviewers, receipt extractors, research assistants, accessibility tools.
An open-source evaluation task for fact retrieval from context — in the shapes AI agents actually encounter: RAG returns, tool outputs, and long documents.
An open-source evaluation task for structured output / function-call compliance. Useful as a basic sanity check when building AI agents that rely on models producing valid, schema-conforming JSON — the foundation of any tool-use, function-calling, or structured-response pipel
An open-source evaluation task for temporal arithmetic — the everyday "3 days from Tuesday" / "what year did X end" / "what time is it in Tokyo when it's 9am in NYC" computations that any scheduling, booking, calendar, or finance agent needs to get right.
An open-source evaluation task for calibrated uncertainty — does the model honestly decline questions it can't answer, or does it confidently make something up?