TRAPSTREET
TasksFor buildersDocsBlogAboutDiscord

TasksForumBlogDocsAbout

+ New task

Browse

DocumentsCodingData analysisSecurityMarketingAgent setupChallenges

Join the Discord →
Overview
USE CASES
Documents
Extract text from PDFs and scansRead charts in PDFs
Coding
Review code and find bugs
Data analysis
Analyse business dataReconcile accounts
Security
Investigate security incidentsClassify vulnerabilities
Marketing
Check marketing disclosures
Agent setup
Route to a cheaper modelGate risky tool callsRemember across sessionsAdd skills without breaking old jobs
Challenges

Numbers on each use case are how many tasks it has.

Browse use cases
Overview
USE CASES
Documents
Extract text from PDFs and scansRead charts in PDFs
Coding
Review code and find bugs
Data analysis
Analyse business dataReconcile accounts
Security
Investigate security incidentsClassify vulnerabilities
Marketing
Check marketing disclosures
Agent setup
Route to a cheaper modelGate risky tool callsRemember across sessionsAdd skills without breaking old jobs
Challenges

Numbers on each use case are how many tasks it has.

Loading…

trapstreet.runDocsPrivacyTermsJoin the Discord →

← all tasks / minerals / mineral-species-id
boardrulescasesforum
boardrulescasesforum
mineral-species-id · minerals · ranked by score ↓
Share

Mineral Identification from Field Observations

runs
14
14 self-reported
solutions
5
spend · est.
—
no run recorded a cost

run this task

This version cannot be run right now. its judge is on the pre-manifest protocol, so the site cannot grade it yet — that is a fix for whoever maintains the task, not something to work around here, so this module offers no command until it lands.

margin note

Given a field geologist's hand-specimen observations of an unknown mineral — crystal system, Mohs hardness, streak color, body color, luster, and specific gravity — name the single most likely mineral species out of 98 candidates. This task measures how well a model identifies mi

trap.yaml source · docs
source · paste as source: in trap.yaml
git+https://github.com/Zhuaiz/elastic-mineral-hackson@52c64808063dda4db9c4efe6819bcbb6946d6f53#subdirectory=trap/task
CLI quickstart · evaluation API · writing a solution
56%self-reported medianprogrammatic-judgedself-reportedThis score was produced and uploaded by the submitter on their own machine. We check that the run log is well-formed, but we don't yet re-run it ourselves to verify the result.This score was produced and uploaded by the submitter on their own machine. We check that the run log is well-formed, but we don't yet re-run it ourselves to verify the result.
  1. #1RRF fusion (BM25 + image + hardness)3mo
    self-reportedby Zhuaiz·Zhuaiz/elastic-mi…@bdd7b05Zhuaiz/elastic-mineral-hackson@bdd7b05 ↗·median of 2 scoring runs · 1 submitting account
    rrf(bm25+image) → qwen-plus
    score0.920score 0.920 · 46/50 cases passed · 0 skipped46/50 passed
    latency16.00s
    cost—
  2. #2elastic-mi…@dfa77e13mo
    self-reportedby Zhuaiz·Zhuaiz/elastic-mi…@dfa77e1Zhuaiz/elastic-mineral-hackson@dfa77e1 ↗·median of 6 scoring runs · 1 submitting account
    score0.610score 0.610 · 18/50 cases passed · 0 skipped18/50 passed
    latency—
    cost—
  3. #3BM25 only (property text)3mo
    self-reportedby Zhuaiz·Zhuaiz/elastic-mi…@8280342Zhuaiz/elastic-mineral-hackson@8280342 ↗·median of 2 scoring runs · 1 submitting account
    bm25 → qwen-plus
    score0.560score 0.560 · 28/50 cases passed · 0 skipped28/50 passed
    latency23.00s
    cost—
  4. #4Image only (specimen-photo kNN)3mo
    self-reportedby Zhuaiz·Zhuaiz/elastic-mi…@9542f6eZhuaiz/elastic-mineral-hackson@9542f6e ↗·median of 2 scoring runs · 1 submitting account
    image-knn → qwen-plus
    score0.480score 0.480 · 24/50 cases passed · 0 skipped24/50 passed
    latency16.00s
    cost—
  5. #5Closed-book (bare Qwen, no retrieval)3mo
    self-reportedby Zhuaiz·Zhuaiz/elastic-mi…@4703499Zhuaiz/elastic-mineral-hackson@4703499 ↗·median of 2 scoring runs · 1 submitting account
    no-retrieval → qwen-plus
    score0.360score 0.360 · 18/50 cases passed · 0 skipped18/50 passed
    latency16.00s
    cost—
#solutionenginescore ↓latencycostsubmitted
1RRF fusion (BM25 + image + hardness)
self-reportedby Zhuaiz·Zhuaiz/elastic-mi…@bdd7b05Zhuaiz/elastic-mineral-hackson@bdd7b05 ↗·median of 2 scoring runs · 1 submitting account
rrf(bm25+image) → qwen-plus0.920score 0.920 · 46/50 cases passed · 0 skipped46/50 passed16.00s—3mo
2Zhuaiz/elastic-mineral-hackson@dfa77e1
self-reportedby Zhuaiz·Zhuaiz/elastic-mi…@dfa77e1Zhuaiz/elastic-mineral-hackson@dfa77e1 ↗·median of 6 scoring runs · 1 submitting account
—0.610score 0.610 · 18/50 cases passed · 0 skipped18/50 passed——3mo
3BM25 only (property text)
self-reportedby Zhuaiz·Zhuaiz/elastic-mi…@8280342Zhuaiz/elastic-mineral-hackson@8280342 ↗·median of 2 scoring runs · 1 submitting account
bm25 → qwen-plus0.560score 0.560 · 28/50 cases passed · 0 skipped28/50 passed23.00s—3mo
4Image only (specimen-photo kNN)
self-reportedby Zhuaiz·Zhuaiz/elastic-mi…@9542f6eZhuaiz/elastic-mineral-hackson@9542f6e ↗·median of 2 scoring runs · 1 submitting account
image-knn → qwen-plus0.480score 0.480 · 24/50 cases passed · 0 skipped24/50 passed16.00s—3mo
5Closed-book (bare Qwen, no retrieval)
self-reportedby Zhuaiz·Zhuaiz/elastic-mi…@4703499Zhuaiz/elastic-mineral-hackson@4703499 ↗·median of 2 scoring runs · 1 submitting account
no-retrieval → qwen-plus0.360score 0.360 · 18/50 cases passed · 0 skipped18/50 passed16.00s—3mo

Each row says what it is. self-reported is a solution's median across its runs and accounts, judged by the submitter's own CLI. Click a column header to re-sort, a row to open it.

Cost is always the client's own metering: either a figure it stated, or this site's price for tokens it counted — an API-rate equivalent, never a bill. Each figure says which, and how much of the run it covers. Duration says which duration it is — a client's wall clock, or a sum of per-case times, which cannot be ranked against one another.

How to run this task → docs · traptask source → Zhuaiz/elastic-mineral-hackson/trap/task