Mineral Identification from Field Observations
ranked by score ↓Source
Paste as source: in your trap.yaml
git+https://github.com/Zhuaiz/elastic-mineral-hackson@dfa77e12bf7db981465d9b54734b985ecd14ec99#subdirectory=trap/taskShare
mineral-species-id
Given a field geologist's hand-specimen observations of an unknown mineral — crystal system, Mohs hardness, streak color, body color, luster, and specific gravity — name the single most likely mineral species out of 98 candidates. This task measures how well a model identifies mi
50 cases
Each case feeds files from inputs/<id>/ to the solution, expects files in expected/<id>/, and is scored by judge.py then aggregated by grader.py.
cases (50)
▸cancriniteIdentify the mineral from hand-specimen properties (cancrinite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system hexagonal; Mohs hardness 5.0-6.0; white streak; color grey-green; vitreous luster; transparent; specific gravity about 2.42.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "cancrinite",
"answer": "cancrinite",
"type": "mineral_species",
"accepted": [
"cancrinit"
],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"cancrinite",
"cancrinit"
]
},
{
"kind": "no_hedge"
}
],
"category": "hexagonal",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸almandineIdentify the mineral from hand-specimen properties (almandine).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system isometric; Mohs hardness 7.0-7.5; white streak; color deep red; vitreous - resinous luster; transparent; specific gravity about 4.318.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "almandine",
"answer": "almandine",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"almandine"
]
},
{
"kind": "no_hedge"
}
],
"category": "isometric",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸grossularIdentify the mineral from hand-specimen properties (grossular).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system isometric; Mohs hardness 6.5-7.0; white to pale streak; color brown; vitreous luster; transparent; specific gravity about 3.594.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "grossular",
"answer": "grossular",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"grossular"
]
},
{
"kind": "no_hedge"
}
],
"category": "isometric",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸fluoriteIdentify the mineral from hand-specimen properties (fluorite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system isometric; Mohs hardness 4.0; white streak; color fluorite displays a wide variety; vitreous luster; transparent; specific gravity about 3.175.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "fluorite",
"answer": "fluorite",
"type": "mineral_species",
"accepted": [
"fluorspar"
],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"fluorite",
"fluorspar"
]
},
{
"kind": "no_hedge"
}
],
"category": "isometric",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸epidoteIdentify the mineral from hand-specimen properties (epidote).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system monoclinic; Mohs hardness 6.0; colourless streak; color yellowish-green; vitreous luster; transparent; specific gravity about 3.38.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "epidote",
"answer": "epidote",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"epidote"
]
},
{
"kind": "no_hedge"
}
],
"category": "monoclinic",
"difficulty": "medium"
}Scored by judge.py — see Scoring logic below for the full rule.
▸chalcopyriteIdentify the mineral from hand-specimen properties (chalcopyrite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system tetragonal; Mohs hardness 3.5-4.0; greenish black streak; color brass yellow; metallic luster; opaque; specific gravity about 4.1.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "chalcopyrite",
"answer": "chalcopyrite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"chalcopyrite"
]
},
{
"kind": "no_hedge"
}
],
"category": "tetragonal",
"difficulty": "easy"
}Scored by judge.py — see Scoring logic below for the full rule.
▸calciteIdentify the mineral from hand-specimen properties (calcite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system trigonal; Mohs hardness 3.0; white streak; color white; vitreous luster; transparent; specific gravity about 2.7102.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "calcite",
"answer": "calcite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"calcite"
]
},
{
"kind": "no_hedge"
}
],
"category": "trigonal",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸berylIdentify the mineral from hand-specimen properties (beryl).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system hexagonal; Mohs hardness 7.5-8.0; white streak; color colorless; vitreous luster; specific gravity about 2.63.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "beryl",
"answer": "beryl",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"beryl"
]
},
{
"kind": "no_hedge"
}
],
"category": "hexagonal",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸pyriteIdentify the mineral from hand-specimen properties (pyrite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system isometric; Mohs hardness 6.0-6.5; greenish-black streak; color pale brass-yellow; metallic luster; opaque; specific gravity about 4.8.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "pyrite",
"answer": "pyrite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"pyrite"
]
},
{
"kind": "no_hedge"
}
],
"category": "isometric",
"difficulty": "easy"
}Scored by judge.py — see Scoring logic below for the full rule.
▸albiteIdentify the mineral from hand-specimen properties (albite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system triclinic; Mohs hardness 6.0-6.5; white streak; color white to gray or colorless; vitreous luster; transparent; specific gravity about 2.6.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "albite",
"answer": "albite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"albite"
]
},
{
"kind": "no_hedge"
}
],
"category": "triclinic",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸aegirineIdentify the mineral from hand-specimen properties (aegirine).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system monoclinic; Mohs hardness 6.0; pale yellowish grey streak; color dark green to greenish black; vitreous luster; transparent; specific gravity about 3.5.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "aegirine",
"answer": "aegirine",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"aegirine"
]
},
{
"kind": "no_hedge"
}
],
"category": "monoclinic",
"difficulty": "easy"
}Scored by judge.py — see Scoring logic below for the full rule.
▸andalusiteIdentify the mineral from hand-specimen properties (andalusite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system orthorhombic; Mohs hardness 6.5-7.5; white streak; color pink to red brown; vitreous luster; transparent; specific gravity about 3.13.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "andalusite",
"answer": "andalusite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"andalusite"
]
},
{
"kind": "no_hedge"
}
],
"category": "orthorhombic",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸titaniteIdentify the mineral from hand-specimen properties (titanite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system monoclinic; Mohs hardness 5.0-5.5; white streak; color brown; adamantine to resinous luster; transparent; specific gravity about 3.48.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "titanite",
"answer": "titanite",
"type": "mineral_species",
"accepted": [
"sphene"
],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"titanite",
"sphene"
]
},
{
"kind": "no_hedge"
}
],
"category": "monoclinic",
"difficulty": "medium"
}Scored by judge.py — see Scoring logic below for the full rule.
▸zirconIdentify the mineral from hand-specimen properties (zircon).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system tetragonal; Mohs hardness 7.5; white streak; color colourless; adamantine luster; transparent; specific gravity about 4.6.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "zircon",
"answer": "zircon",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"zircon"
]
},
{
"kind": "no_hedge"
}
],
"category": "tetragonal",
"difficulty": "medium"
}Scored by judge.py — see Scoring logic below for the full rule.
▸galenaIdentify the mineral from hand-specimen properties (galena).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system isometric; Mohs hardness 2.5; lead-grey streak; color lead-grey; metallic luster; opaque; specific gravity about 7.60.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "galena",
"answer": "galena",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"galena"
]
},
{
"kind": "no_hedge"
}
],
"category": "isometric",
"difficulty": "easy"
}Scored by judge.py — see Scoring logic below for the full rule.
▸hematiteIdentify the mineral from hand-specimen properties (hematite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system trigonal; Mohs hardness 5.0-6.0; reddish brown ("rust-red") streak; color steel-grey to black in crystals; metallic luster; opaque; specific gravity about 5.26.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "hematite",
"answer": "hematite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"hematite"
]
},
{
"kind": "no_hedge"
}
],
"category": "trigonal",
"difficulty": "easy"
}Scored by judge.py — see Scoring logic below for the full rule.
▸sphaleriteIdentify the mineral from hand-specimen properties (sphalerite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system isometric; Mohs hardness 3.5-4.0; pale yellow to streak; color mostly light to dark brown; resinous luster; transparent; specific gravity about 3.9.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "sphalerite",
"answer": "sphalerite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"sphalerite"
]
},
{
"kind": "no_hedge"
}
],
"category": "isometric",
"difficulty": "easy"
}Scored by judge.py — see Scoring logic below for the full rule.
▸vivianiteIdentify the mineral from hand-specimen properties (vivianite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system monoclinic; Mohs hardness 1.5-2.0; colourless to bluish streak; color colourless and transparent when fresh; vitreous luster; transparent; specific gravity about 2.67.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "vivianite",
"answer": "vivianite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"vivianite"
]
},
{
"kind": "no_hedge"
}
],
"category": "monoclinic",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸bournoniteIdentify the mineral from hand-specimen properties (bournonite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system orthorhombic; Mohs hardness 2.5-3.0; steel-gray streak; color steel-gray; metallic luster; opaque; specific gravity about 5.83.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "bournonite",
"answer": "bournonite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"bournonite"
]
},
{
"kind": "no_hedge"
}
],
"category": "orthorhombic",
"difficulty": "easy"
}Scored by judge.py — see Scoring logic below for the full rule.
▸molybdeniteIdentify the mineral from hand-specimen properties (molybdenite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system hexagonal; Mohs hardness 1.0-1.5; bluish gray streak; color black; metallic luster; opaque; specific gravity about 4.62.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "molybdenite",
"answer": "molybdenite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"molybdenite"
]
},
{
"kind": "no_hedge"
}
],
"category": "hexagonal",
"difficulty": "easy"
}Scored by judge.py — see Scoring logic below for the full rule.
▸magnetiteIdentify the mineral from hand-specimen properties (magnetite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system isometric; Mohs hardness 5.5-6.5; black streak; color greyish black or iron black; metallic luster; opaque; specific gravity about 5.175.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "magnetite",
"answer": "magnetite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"magnetite"
]
},
{
"kind": "no_hedge"
}
],
"category": "isometric",
"difficulty": "easy"
}Scored by judge.py — see Scoring logic below for the full rule.
▸microclineIdentify the mineral from hand-specimen properties (microcline).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system triclinic; Mohs hardness 6.0-6.5; white streak; color white; vitreous luster; transparent; specific gravity about 2.54.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "microcline",
"answer": "microcline",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"microcline"
]
},
{
"kind": "no_hedge"
}
],
"category": "triclinic",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸graphiteIdentify the mineral from hand-specimen properties (graphite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system hexagonal; Mohs hardness 1.0-2.0; black to steel streak; color iron black to steel-grey; sub-metallic to earthy luster; opaque; specific gravity about 2.09.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "graphite",
"answer": "graphite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"graphite"
]
},
{
"kind": "no_hedge"
}
],
"category": "hexagonal",
"difficulty": "easy"
}Scored by judge.py — see Scoring logic below for the full rule.
▸dolomiteIdentify the mineral from hand-specimen properties (dolomite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system trigonal; Mohs hardness 3.5-4.0; white streak; color colourless; vitreous luster; transparent; specific gravity about 2.84.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "dolomite",
"answer": "dolomite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"dolomite"
]
},
{
"kind": "no_hedge"
}
],
"category": "trigonal",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸rutileIdentify the mineral from hand-specimen properties (rutile).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system tetragonal; Mohs hardness 6.0-6.5; greyish black streak; color blood red; adamantine luster; transparent; specific gravity about 4.23.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "rutile",
"answer": "rutile",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"rutile"
]
},
{
"kind": "no_hedge"
}
],
"category": "tetragonal",
"difficulty": "easy"
}Scored by judge.py — see Scoring logic below for the full rule.
▸tremoliteIdentify the mineral from hand-specimen properties (tremolite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system monoclinic; Mohs hardness 5.0-6.0; white streak; color white; vitreous luster; transparent; specific gravity about 2.99.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "tremolite",
"answer": "tremolite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"tremolite"
]
},
{
"kind": "no_hedge"
}
],
"category": "monoclinic",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸hedenbergiteIdentify the mineral from hand-specimen properties (hedenbergite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system monoclinic; Mohs hardness 5.5-6.5; white streak; color brownish green; vitreous luster; transparent; specific gravity about 3.56.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "hedenbergite",
"answer": "hedenbergite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"hedenbergite"
]
},
{
"kind": "no_hedge"
}
],
"category": "monoclinic",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸topazIdentify the mineral from hand-specimen properties (topaz).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system orthorhombic; Mohs hardness 8.0; white streak; color colourless; vitreous luster; transparent; specific gravity about 3.4.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "topaz",
"answer": "topaz",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"topaz"
]
},
{
"kind": "no_hedge"
}
],
"category": "orthorhombic",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸actinoliteIdentify the mineral from hand-specimen properties (actinolite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system monoclinic; Mohs hardness 5.0-6.0; white streak; color green; vitreous luster; transparent; specific gravity about 3.03.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "actinolite",
"answer": "actinolite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"actinolite"
]
},
{
"kind": "no_hedge"
}
],
"category": "monoclinic",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸azuriteIdentify the mineral from hand-specimen properties (azurite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system monoclinic; Mohs hardness 3.5-4.0; light blue streak; color azure blue; vitreous luster; transparent; specific gravity about 3.77.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "azurite",
"answer": "azurite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"azurite"
]
},
{
"kind": "no_hedge"
}
],
"category": "monoclinic",
"difficulty": "easy"
}Scored by judge.py — see Scoring logic below for the full rule.
▸elbaiteIdentify the mineral from hand-specimen properties (elbaite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system trigonal; Mohs hardness 7.5; white streak; color green; vitreous to oily luster; transparent; specific gravity about 2.9.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "elbaite",
"answer": "elbaite",
"type": "mineral_species",
"accepted": [
"elbait"
],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"elbaite",
"elbait"
]
},
{
"kind": "no_hedge"
}
],
"category": "trigonal",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸cobaltiteIdentify the mineral from hand-specimen properties (cobaltite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system orthorhombic; Mohs hardness 5.5; grayish black streak; color reddish silver white; metallic luster; opaque; specific gravity about 6.33.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "cobaltite",
"answer": "cobaltite",
"type": "mineral_species",
"accepted": [
"cobaltin"
],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"cobaltite",
"cobaltin"
]
},
{
"kind": "no_hedge"
}
],
"category": "orthorhombic",
"difficulty": "easy"
}Scored by judge.py — see Scoring logic below for the full rule.
▸polluciteIdentify the mineral from hand-specimen properties (pollucite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system isometric; Mohs hardness 6.5-7.0; white streak; color almost always colorless or white; vitreous luster; transparent; specific gravity about 2.68.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "pollucite",
"answer": "pollucite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"pollucite"
]
},
{
"kind": "no_hedge"
}
],
"category": "isometric",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸augiteIdentify the mineral from hand-specimen properties (augite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system monoclinic; Mohs hardness 5.5-6.0; greenish gray streak; color brown-green; vitreous luster; translucent; specific gravity about 3.19.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "augite",
"answer": "augite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"augite"
]
},
{
"kind": "no_hedge"
}
],
"category": "monoclinic",
"difficulty": "easy"
}Scored by judge.py — see Scoring logic below for the full rule.
▸scheeliteIdentify the mineral from hand-specimen properties (scheelite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system tetragonal; Mohs hardness 4.5-5.0; white streak; color tan; vitreous luster; transparent; specific gravity about 6.1.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "scheelite",
"answer": "scheelite",
"type": "mineral_species",
"accepted": [
"scheelit"
],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"scheelite",
"scheelit"
]
},
{
"kind": "no_hedge"
}
],
"category": "tetragonal",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸kyaniteIdentify the mineral from hand-specimen properties (kyanite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system triclinic; Mohs hardness 5.5-7.0; colorless streak; color blue; vitreous luster; transparent; specific gravity about 3.53.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "kyanite",
"answer": "kyanite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"kyanite"
]
},
{
"kind": "no_hedge"
}
],
"category": "triclinic",
"difficulty": "easy"
}Scored by judge.py — see Scoring logic below for the full rule.
▸andraditeIdentify the mineral from hand-specimen properties (andradite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system isometric; Mohs hardness 6.5-7.0; white streak; color yellow; vitreous luster; transparent; specific gravity about 3.8.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "andradite",
"answer": "andradite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"andradite"
]
},
{
"kind": "no_hedge"
}
],
"category": "isometric",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸sideriteIdentify the mineral from hand-specimen properties (siderite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system trigonal; Mohs hardness 3.5-4.5; white streak; color yellowish-brown to greyish-brown; vitreous luster; translucent; specific gravity about 3.96.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "siderite",
"answer": "siderite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"siderite"
]
},
{
"kind": "no_hedge"
}
],
"category": "trigonal",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸cupriteIdentify the mineral from hand-specimen properties (cuprite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system isometric; Mohs hardness 3.5-4.0; shining metallic brownish-red streak; color dark red to cochineal red; adamantine to sub-metallic luster; transparent; specific gravity about 6.14.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "cuprite",
"answer": "cuprite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"cuprite"
]
},
{
"kind": "no_hedge"
}
],
"category": "isometric",
"difficulty": "easy"
}Scored by judge.py — see Scoring logic below for the full rule.
▸spodumeneIdentify the mineral from hand-specimen properties (spodumene).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system monoclinic; Mohs hardness 6.5-7.0; white streak; color colourless; vitreous luster; transparent; specific gravity about 3.1.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "spodumene",
"answer": "spodumene",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"spodumene"
]
},
{
"kind": "no_hedge"
}
],
"category": "monoclinic",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸corundumIdentify the mineral from hand-specimen properties (corundum).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system trigonal; Mohs hardness 9.0; white streak; color colourless; adamantine luster; transparent; specific gravity about 3.98.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "corundum",
"answer": "corundum",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"corundum"
]
},
{
"kind": "no_hedge"
}
],
"category": "trigonal",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸prehniteIdentify the mineral from hand-specimen properties (prehnite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system orthorhombic; Mohs hardness 6.0-6.5; white streak; color colourless to gray to yellow; vitreous to pearly luster; translucent; specific gravity about 2.8.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "prehnite",
"answer": "prehnite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"prehnite"
]
},
{
"kind": "no_hedge"
}
],
"category": "orthorhombic",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸stibniteIdentify the mineral from hand-specimen properties (stibnite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system orthorhombic; Mohs hardness 2.0; lead grey streak; color lead-gray with pale blue tint; metallic luster; opaque; specific gravity about 4.63.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "stibnite",
"answer": "stibnite",
"type": "mineral_species",
"accepted": [
"antimonite"
],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"stibnite",
"antimonite"
]
},
{
"kind": "no_hedge"
}
],
"category": "orthorhombic",
"difficulty": "easy"
}Scored by judge.py — see Scoring logic below for the full rule.
▸chrysoberylIdentify the mineral from hand-specimen properties (chrysoberyl).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system orthorhombic; Mohs hardness 8.5; white streak; color green shades; vitreous luster; transparent; specific gravity about 3.75.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "chrysoberyl",
"answer": "chrysoberyl",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"chrysoberyl"
]
},
{
"kind": "no_hedge"
}
],
"category": "orthorhombic",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸arsenopyriteIdentify the mineral from hand-specimen properties (arsenopyrite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system monoclinic; Mohs hardness 5.5-6.0; gray-black streak; color silver-white to steel-gray; metallic luster; opaque; specific gravity about 6.07.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "arsenopyrite",
"answer": "arsenopyrite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"arsenopyrite"
]
},
{
"kind": "no_hedge"
}
],
"category": "monoclinic",
"difficulty": "easy"
}Scored by judge.py — see Scoring logic below for the full rule.
▸diopsideIdentify the mineral from hand-specimen properties (diopside).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system monoclinic; Mohs hardness 5.5-6.5; white streak; color light to dark green; vitreous luster; transparent; specific gravity about 3.22.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "diopside",
"answer": "diopside",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"diopside"
]
},
{
"kind": "no_hedge"
}
],
"category": "monoclinic",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸pyrrhotiteIdentify the mineral from hand-specimen properties (pyrrhotite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system monoclinic; Mohs hardness 3.5-4.0; dark grayish black streak; color bronze brown; metallic luster; opaque; specific gravity about 4.58.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "pyrrhotite",
"answer": "pyrrhotite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"pyrrhotite"
]
},
{
"kind": "no_hedge"
}
],
"category": "monoclinic",
"difficulty": "easy"
}Scored by judge.py — see Scoring logic below for the full rule.
▸creediteIdentify the mineral from hand-specimen properties (creedite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system monoclinic; Mohs hardness 4.0; white streak; color white; vitreous luster; transparent; specific gravity about 2.713.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "creedite",
"answer": "creedite",
"type": "mineral_species",
"accepted": [
"credit"
],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"creedite",
"credit"
]
},
{
"kind": "no_hedge"
}
],
"category": "monoclinic",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸natroliteIdentify the mineral from hand-specimen properties (natrolite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system orthorhombic; Mohs hardness 5.0-5.5; white streak; color white; vitreous luster; transparent; specific gravity about 2.2.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "natrolite",
"answer": "natrolite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"natrolite"
]
},
{
"kind": "no_hedge"
}
],
"category": "orthorhombic",
"difficulty": "hard"
}Scored by judge.py — see Scoring logic below for the full rule.
▸ilmeniteIdentify the mineral from hand-specimen properties (ilmenite).
input
question.txt
A field geologist recorded these hand-specimen observations of an unknown mineral:
crystal system trigonal; Mohs hardness 5.0-6.0; black to reddish streak; color iron black or black; sub metallic luster; opaque; specific gravity about 4.68.
Identify the mineral species. Choose from:
actinolite, aegirine, agate, albite, almandine, amazonite, amber, amethyst, analcime, andalusite, andradite, apatite, aragonite, arsenic, arsenopyrite, augite, azurite, barite, beryl, bismuth, bournonite, calcite, cancrinite, carnelian, cassiterite, celestine, chalcedony, chalcopyrite, chromite, chrysoberyl, chrysoprase, cinnabar, cobaltite, copper, corundum, creedite, cuprite, diopside, dolomite, elbaite, epidote, eudialyte, flint, fluorapatite, fluorite, galena, goethite, gold, graphite, grossular, gypsum, hedenbergite, hematite, hornblende, ilmenite, jasper, kyanite, labradorite, lapis lazuli, limonite, magnetite, malachite, marcasite, microcline, molybdenite, muscovite, natrolite, nephrite, oligoclase, opal, orthoclase, pectolite, pollucite, prehnite, proustite, pyrargyrite, pyrite, pyrrhotite, quartz, rhodonite, rutile, scheelite, siderite, silver, sphalerite, spinel, spodumene, stibnite, sulfur, tetrahedrite, titanite, topaz, tourmaline, tremolite, vesuvianite, vivianite, wollastonite, zircon
Answer with the exact species name from the list only, one lowercase word. Do not hedge or explain.expected output
answer.json
{
"id": "ilmenite",
"answer": "ilmenite",
"type": "mineral_species",
"accepted": [],
"matchers": [
{
"kind": "keywords_any_word",
"values": [
"ilmenite"
]
},
{
"kind": "no_hedge"
}
],
"category": "trigonal",
"difficulty": "easy"
}Scored by judge.py — see Scoring logic below for the full rule.
scoring logic
judge.py runs once per case and prints a score per case. grader.py runs once at the end and folds case scores into a run-level summary. Without grader.py, the run's score is simply the average of case scores.
▸judge.py135 lines · view on GitHub
"""Per-case judge for the mineral-id task (trapstreet contract).
Reads the payload from $TRAPTASK_PAYLOAD, extracts the agent's answer, and
scores 1.0 iff the answer names the correct mineral species. Species names are
normalized (lowercase, punctuation stripped, known synonyms/transliterations
collapsed — creedite/credit, stibnite/antimonite, labradorite/labrador, ...)
so a model answering with a common variant still passes. Hedging answers
("I cannot determine ...") score 0.
Payload contract (same as trapstreet/trapstreet-tasks tasks):
outputs.case_stdout -> agent answer (plain text OR {"answer": "..."})
outputs.case_meta.json -> {"exit_code": int}
expected.answer.json -> {id, answer, accepted[], category, difficulty}
Emits JSON metrics on stdout: {score, agent_answer, expected_answer, ...}.
"""
from __future__ import annotations
import json
import os
import re
from pathlib import Path
# 同义词/变体 -> 规范名（与 make_cases.species_synonyms 保持一致，双向收敛）
SYNONYMS = {
"antimonite": "stibnite", "stibnite": "stibnite",
"labrador": "labradorite", "labradorite": "labradorite",
"nephritis": "nephrite", "nephrite": "nephrite",
"credit": "creedite", "creedite": "creedite",
"cobaltin": "cobaltite", "cobaltite": "cobaltite",
"analcim": "analcime", "analcite": "analcime", "analcime": "analcime",
"cancrinit": "cancrinite", "cancrinite": "cancrinite",
"elbait": "elbaite", "elbaite": "elbaite",
"scheelit": "scheelite", "scheelite": "scheelite",
"vesuvian": "vesuvianite", "idocrase": "vesuvianite", "vesuvianite": "vesuvianite",
"sphene": "titanite", "titanite": "titanite",
"fluorspar": "fluorite", "fluorite": "fluorite",
"baryte": "barite", "barite": "barite",
}
HEDGES = [
"i cannot", "i can't", "i am unable", "i'm unable", "cannot determine",
"unable to determine", "i don't know", "i do not know", "as an ai",
"not enough information", "insufficient information", "unclear",
]
def normalize(s: str) -> str:
s = s.strip().lower()
s = re.sub(r"^\s*(?:answer|species|a)\s*[:\-]\s*", "", s) # 去 "Answer:" 前缀
s = re.sub(r"[^a-z\s-]", "", s)
first = s.split("\n")[0].strip()
# 取词元里第一个已知种名（模型可能写 "the mineral is malachite"）
tokens = [t for t in re.split(r"\s+", first) if t]
for t in tokens:
if t in SYNONYMS:
return SYNONYMS[t]
tok = tokens[0] if tokens else ""
return SYNONYMS.get(tok, tok)
def extract_answer(stdout: str) -> str:
stdout = stdout.strip()
if not stdout:
return ""
try:
obj = json.loads(stdout)
if isinstance(obj, dict) and "answer" in obj:
return str(obj["answer"])
except json.JSONDecodeError:
pass
return stdout
def is_hedge(text: str) -> bool:
low = re.sub(r"\s+", " ", text).strip().lower()
return any(h in low for h in HEDGES)
def accepted_forms(expected: dict) -> set[str]:
"""所有可接受的书写形式（规范名 + 声明的同义词 + 它们的原始拼写）。"""
forms = {normalize(expected["answer"]), expected["answer"].strip().lower()}
for a in expected.get("accepted") or []:
forms.add(normalize(a))
forms.add(a.strip().lower())
return forms
def whole_word_hit(answer: str, forms: set[str]) -> bool:
"""短答案里，任一可接受形式作为完整词出现即算命中（对齐 keywords_any_word）。
限制词数以防模型把候选清单整个 dump 出来蒙混。"""
low = re.sub(r"[^a-z\s-]", " ", answer.lower())
if len(low.split()) > 8:
return False
return any(re.search(rf"\b{re.escape(f)}\b", low) for f in forms if f)
def main() -> None:
payload = json.loads(os.environ["TRAPTASK_PAYLOAD"])
stdout = Path(payload["outputs"]["case_stdout"]).read_text()
exit_code = json.loads(Path(payload["outputs"]["case_meta.json"]).read_text())["exit_code"]
expected = json.loads(Path(payload["expected"]["answer.json"]).read_text())
base = {
"agent_answer": extract_answer(stdout),
"expected_answer": expected.get("answer"),
"id": expected.get("id"),
"category": expected.get("category"),
"difficulty": expected.get("difficulty"),
}
if exit_code != 0:
print(json.dumps({"score": 0.0, "reason": f"solution exited {exit_code}", **base}))
return
answer = base["agent_answer"]
if not answer:
print(json.dumps({"score": 0.0, "reason": "empty answer", **base}))
return
if is_hedge(answer):
print(json.dumps({"score": 0.0, "reason": "hedged answer", **base}))
return
forms = accepted_forms(expected)
got = normalize(answer)
ok = got in forms or whole_word_hit(answer, forms)
print(json.dumps({
"score": 1.0 if ok else 0.0,
"reason": f"answer {answer.strip()[:40]!r} " + ("matched" if ok else "≠")
+ f" {normalize(expected['answer'])!r}",
**base,
}))
if __name__ == "__main__":
main()
▸grader.py62 lines · view on GitHub
"""Overall grader for the mineral-id task (trapstreet contract).
Aggregates per-case judge results into a run-level verdict. Reads the case list
from $TRAPTASK_PAYLOAD and emits JSON on stdout with `passed`, `score`
(accuracy), per-category breakdown, and latency stats — the leaderboard reads
these. Pass threshold = 60% (fine-grained 98-way mineral ID is hard).
"""
from __future__ import annotations
import json
import os
from collections import Counter
PASS_THRESHOLD = 0.60
def main() -> None:
cases = json.loads(os.environ["TRAPTASK_PAYLOAD"])
scored = [c for c in cases if c.get("metrics") and c["metrics"].get("score") is not None]
skipped = [c for c in cases if not c.get("metrics") or c["metrics"].get("score") is None]
accuracy = (sum(c["metrics"]["score"] for c in scored) / len(scored)) if scored else 0.0
by_score: Counter[str] = Counter()
by_total: Counter[str] = Counter()
for c in scored:
cat = c["metrics"].get("difficulty") or c["metrics"].get("category")
if cat:
by_total[cat] += 1
by_score[cat] += c["metrics"]["score"]
by_category_pct = {k: round(by_score[k] / by_total[k], 3) for k in by_total}
durations = [c.get("duration", 0.0) for c in cases if c.get("duration") is not None]
if durations:
ds = sorted(durations)
latency_ms_median = round(ds[len(ds) // 2] * 1000, 1)
latency_ms_p95 = round(ds[int(0.95 * len(ds))] * 1000, 1) if len(ds) > 1 else latency_ms_median
latency_ms_total = round(sum(ds) * 1000, 1)
else:
latency_ms_median = latency_ms_p95 = latency_ms_total = 0.0
n_passed = sum(1 for c in scored if c["metrics"]["score"] == 1.0)
print(json.dumps({
"passed": bool(scored) and accuracy >= PASS_THRESHOLD,
"score": round(accuracy, 3),
"n_passed": n_passed,
"n_total": len(cases),
"n_scored": len(scored),
"n_skipped_no_gold": len(skipped),
"threshold": PASS_THRESHOLD,
"by_category": by_category_pct,
"latency_ms_median": latency_ms_median,
"latency_ms_p95": latency_ms_p95,
"latency_ms_total": latency_ms_total,
"cost_usd_total": None,
}))
if __name__ == "__main__":
main()