LLM Plays REAL Minecraft - Can Your AI Mine a Diamond? 💎

The task

Starting from a fresh survival world with nothing, an agent must work its way down the tech tree — wood → stone tools → iron (mine + smelt) → diamond — and end with at least one diamond in its inventory, within a time limit. This is the classic long-horizon Minecraft challenge: many dependent steps, tool-gated progression, and no shortcuts.

Goal & rules

Obtain ≥ 1 diamond. Fix these so runs are comparable:

What you submit

Your agent plays the run and produces two things:

1. An outcome — a single JSON object printed as the last line of stdout:

   {
     "obtained": true,
     "item": "diamond",
     "count": 1,
     "ticks": 41234,
     "wall_time_s": 512.3,
     "video": "https://.../recording.mp4",
     "seed": "your-seed",
     "mc_version": "1.20.4"
   }
   

2. A public run video — a recording of the whole run (the video field links to it). The video is required: it's both the fun part and the credibility check.

How it's scored

Scoring is done by the scripts in this repo (judge.py + grader.py):

• Per run (judge.py) — graded by tech-tree progress (highest rung wins): wooden pickaxe 0.2 · stone pickaxe 0.4 · iron ingot 0.6 · iron pickaxe 0.8 · diamond 1.0. A non-empty video link is required — no video ⇒ 0.0, however far the run got. ticks / wall_time_s are surfaced for speed ranking.

• Ranking — score first, then speed (fewer ticks / less wall-clock wins).

  Reaching a diamond is genuinely hard, so partial credit lets the task distinguish an agent that gets nowhere from one that nearly made it.

The current version trusts the reported outcome and leans on the public video for credibility. An automatic verifier (deterministic replay of the run against the game's real rules) is planned so runs can be objectively re-checked rather than self-reported — but the video is what makes a claim believable today. A video is credibility, not proof: it can be faked or edited, which is exactly what the future verifier is for.

How to participate

1. Get an agent. Write one, or adapt an existing Minecraft agent (Voyager, Mindcraft, or the reference agents in the solution repo). It needs to drive a Minecraft bot toward the goal, record the run, and print the outcome JSON.
2. Stand up a server. A Minecraft Java 1.20.4 server in offline mode with a fixed seed (setup in the solution repo).
3. Run it. The agent plays, records to an mp4, and prints the outcome JSON.
4. Score it. Feed the run's stdout + goal spec through judge.py to get your score, then publish your video and outcome.

Repo layout

This repo is the task — the gradeable definition, at the repo root so a trap git+ source resolves it directly. The reference agents and recorder live in a separate solution repo (task and solution are kept apart).

• traptask.yaml — the trap task config (cases, judge/grader cmds).
• judge.py — per-run scoring by tech-tree progress. grader.py — aggregation.
• inputs/obtain_diamond/spec.md — the full rules. expected/obtain_diamond/expected.json — the milestone ladder.
• mindcraft-forMC — the reference solution.

Status

The task definition and scoring are complete and unit-tested. See the solution repo for the runnable agents and their status.