Share
⚡ Jev as a router: cheaper than always-Opus?
- runs
- 6
- 6 graded on site
- solutions
- 0
- 6 single agent runs graded on site
- spend · est.
- $9.12
- $9.12 declared · not a bill
run this task
200 cases
Each case feeds files from inputs/<id>/ to the solution, expects files in expected/<id>/. This pinned commit declares no judge, so the scoring code is not here — a task can publish its questions publicly and keep the reference answers and the judge in a private grading pack.
cases (200)
▸case_001Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A housing corporation owned a tract of land. The housing corporation prepared a development plan to divide the land into 100 lots and create a residential community on the tract. The Declaration of Covenants created the homeowners' association, an administrative entity that would administer and enforce the regulations and restrictions recited among the covenants. One of the restrictions set forth in the Declaration of Covenants reads:\\\"There shall never at any time be erected, permitted, or maintained upon any part of the property any structure designed for or used as a saloon or place for the sale or manufacture of malt, vinous, or spirituous liquors. \\\"The Declaration of Covenants was duly recorded and was included in the deed taken by a teacher when he purchased lot 62 in the development. The teacher immediately recorded his deed. The teacher leased his home to a librarian for a term of one year. The lease included the same restrictions as those in the Declaration of Covenants and in the teacher's deed. The librarian immediately began to sell liquor on the premises during weekly \\\"after hours\\\" parties. The homeowners' association sues the librarian in an action for damages. Which of the following is the best argument for the librarian?\",\n \"options\": {\n \"A\": \"The homeowners' association has no authority over the librarian.\",\n \"B\": \"The librarian is not in privity of contract with the homeowners' association.\",\n \"C\": \"The homeowners' association failed to notify the librarian of the restriction.\",\n \"D\": \"The librarian has the right to use the property as they see fit during the lease term.\",\n \"E\": \"The restriction is considered void due to the other commercial activities in the vicinity.\",\n \"F\": \"The Declaration of Covenants restriction is not legally binding.\",\n \"G\": \"Other lots in the immediate vicinity are used for commercial purposes.\",\n \"H\": \"The librarian was not aware of the Declaration of Covenants when signing the lease.\",\n \"I\": \"The rule in Spencer's Case prevents the librarian from being liable.\",\n \"J\": \"The librarian is not in privity of estate with the teacher.\"\n }\n}
input
question.json
{
"question": "A housing corporation owned a tract of land. The housing corporation prepared a development plan to divide the land into 100 lots and create a residential community on the tract. The Declaration of Covenants created the homeowners' association, an administrative entity that would administer and enforce the regulations and restrictions recited among the covenants. One of the restrictions set forth in the Declaration of Covenants reads:\"There shall never at any time be erected, permitted, or maintained upon any part of the property any structure designed for or used as a saloon or place for the sale or manufacture of malt, vinous, or spirituous liquors. \"The Declaration of Covenants was duly recorded and was included in the deed taken by a teacher when he purchased lot 62 in the development. The teacher immediately recorded his deed. The teacher leased his home to a librarian for a term of one year. The lease included the same restrictions as those in the Declaration of Covenants and in the teacher's deed. The librarian immediately began to sell liquor on the premises during weekly \"after hours\" parties. The homeowners' association sues the librarian in an action for damages. Which of the following is the best argument for the librarian?",
"options": {
"A": "The homeowners' association has no authority over the librarian.",
"B": "The librarian is not in privity of contract with the homeowners' association.",
"C": "The homeowners' association failed to notify the librarian of the restriction.",
"D": "The librarian has the right to use the property as they see fit during the lease term.",
"E": "The restriction is considered void due to the other commercial activities in the vicinity.",
"F": "The Declaration of Covenants restriction is not legally binding.",
"G": "Other lots in the immediate vicinity are used for commercial purposes.",
"H": "The librarian was not aware of the Declaration of Covenants when signing the lease.",
"I": "The rule in Spencer's Case prevents the librarian from being liable.",
"J": "The librarian is not in privity of estate with the teacher."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_001/ files.
Scored by a judge this pinned commit does not carry.
▸case_002Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A television picture is composed of approximately 300,000 individual picture elements, each of which can attain ten different and distinguishable brightness levels (such as black and shades of gray) for proper contrast. If, for any picture element, the probability of occurrence of the ten brightness levels are equal, and 30 picture frames are being transmitted per second, determine the bandwidth of the T.V. video signal. Assume that the signal-to-noise ratio of 1000 (30 db) is required for proper reproduction of any picture.\",\n \"options\": {\n \"A\": \"2.5MHz\",\n \"B\": \"1.5MHz\",\n \"C\": \"6MHz\",\n \"D\": \"8MHz\",\n \"E\": \"2MHz\",\n \"F\": \"4MHz\",\n \"G\": \"3MHz\",\n \"H\": \"1MHz\",\n \"I\": \"7MHz\",\n \"J\": \"5MHz\"\n }\n}
input
question.json
{
"question": "A television picture is composed of approximately 300,000 individual picture elements, each of which can attain ten different and distinguishable brightness levels (such as black and shades of gray) for proper contrast. If, for any picture element, the probability of occurrence of the ten brightness levels are equal, and 30 picture frames are being transmitted per second, determine the bandwidth of the T.V. video signal. Assume that the signal-to-noise ratio of 1000 (30 db) is required for proper reproduction of any picture.",
"options": {
"A": "2.5MHz",
"B": "1.5MHz",
"C": "6MHz",
"D": "8MHz",
"E": "2MHz",
"F": "4MHz",
"G": "3MHz",
"H": "1MHz",
"I": "7MHz",
"J": "5MHz"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_002/ files.
Scored by a judge this pinned commit does not carry.
▸case_003Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"(i) A certain 230-volt motor has an armature-circuit resistance of 0.3 ohm and runs at a speed of 1200 rpm, with normal excitation and 50-amp armature current. (a) If the load is increased until the armature current has risen to 60 amp, what will be the new value of speed ? (b) If the load is reduced until the armature current has dropped to 25 amp, what will the speed become ? (ii) (a) If an attempt is made to start the motor of part (i) without auxiliary resistance, what will be the value of the current at the instant of closing the switch ? (b) What series resistance should be inserted in order that the armature current at the moment of starting may be limited to 1.5 times the normal value of 50 amp ?\",\n \"options\": {\n \"A\": \"S_2 = 1200 rpm, S_3 = 1200 rpm, I_a = 766 amp, R_s = 3 ohms\",\n \"B\": \"S_2 = 1100 rpm, S_3 = 1320 rpm, I_a = 765 amp, R_s = 2.8 ohms\",\n \"C\": \"S_2 = 1210 rpm, S_3 = 1180 rpm, I_a = 800 amp, R_s = 3.2 ohms\",\n \"D\": \"S_2 = 1250 rpm, S_3 = 1150 rpm, I_a = 770 amp, R_s = 3.5 ohms\",\n \"E\": \"S_2 = 1150 rpm, S_3 = 1275 rpm, I_a = 750 amp, R_s = 2.2 ohms\",\n \"F\": \"S_2 = 1230 rpm, S_3 = 1165 rpm, I_a = 755 amp, R_s = 3.1 ohms\",\n \"G\": \"S_2 = 1175 rpm, S_3 = 1225 rpm, I_a = 780 amp, R_s = 2.9 ohms\",\n \"H\": \"S_2 = 1130 rpm, S_3 = 1300 rpm, I_a = 760 amp, R_s = 2.5 ohms\",\n \"I\": \"S_2 = 1183 rpm, S_3 = 1242 rpm, I_a = 766 (2/3) amp, R_s = 2.77 ohms\",\n \"J\": \"S_2 = 1190 rpm, S_3 = 1210 rpm, I_a = 765 amp, R_s = 2.6 ohms\"\n }\n}
input
question.json
{
"question": "(i) A certain 230-volt motor has an armature-circuit resistance of 0.3 ohm and runs at a speed of 1200 rpm, with normal excitation and 50-amp armature current. (a) If the load is increased until the armature current has risen to 60 amp, what will be the new value of speed ? (b) If the load is reduced until the armature current has dropped to 25 amp, what will the speed become ? (ii) (a) If an attempt is made to start the motor of part (i) without auxiliary resistance, what will be the value of the current at the instant of closing the switch ? (b) What series resistance should be inserted in order that the armature current at the moment of starting may be limited to 1.5 times the normal value of 50 amp ?",
"options": {
"A": "S_2 = 1200 rpm, S_3 = 1200 rpm, I_a = 766 amp, R_s = 3 ohms",
"B": "S_2 = 1100 rpm, S_3 = 1320 rpm, I_a = 765 amp, R_s = 2.8 ohms",
"C": "S_2 = 1210 rpm, S_3 = 1180 rpm, I_a = 800 amp, R_s = 3.2 ohms",
"D": "S_2 = 1250 rpm, S_3 = 1150 rpm, I_a = 770 amp, R_s = 3.5 ohms",
"E": "S_2 = 1150 rpm, S_3 = 1275 rpm, I_a = 750 amp, R_s = 2.2 ohms",
"F": "S_2 = 1230 rpm, S_3 = 1165 rpm, I_a = 755 amp, R_s = 3.1 ohms",
"G": "S_2 = 1175 rpm, S_3 = 1225 rpm, I_a = 780 amp, R_s = 2.9 ohms",
"H": "S_2 = 1130 rpm, S_3 = 1300 rpm, I_a = 760 amp, R_s = 2.5 ohms",
"I": "S_2 = 1183 rpm, S_3 = 1242 rpm, I_a = 766 (2/3) amp, R_s = 2.77 ohms",
"J": "S_2 = 1190 rpm, S_3 = 1210 rpm, I_a = 765 amp, R_s = 2.6 ohms"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_003/ files.
Scored by a judge this pinned commit does not carry.
▸case_004Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Suppose a fluid particle moves according to the equations x = at, y =bt- ct^2, where a, b, c are constants. Find the normal and tangential accelerations.\",\n \"options\": {\n \"A\": \"a_n = [(2ca) / \\\\surd{a^2 + b^2}], a_t = [{2c(b - ct)} / \\\\surd{a^2 + b^2}]\",\n \"B\": \"a_n = [(2cb) / \\\\surd{a^2 + (b + 2ct)^2}], a_t = [{2c(a + 2ct)} / \\\\surd{a^2 + (b + 2ct)^2}]\",\n \"C\": \"a_n = [(2ca) / \\\\surd{(a - 2ct)^2 + b^2}], a_t = [{2c(b - ct)} / \\\\surd{(a - 2ct)^2 + b^2}]\",\n \"D\": \"a_n = [c / \\\\surd{a^2 + (b - 2ct)^2}], a_t = [{2c(b - ct)} / \\\\surd{a^2 + (b - 2ct)^2}]\",\n \"E\": \"a_n = [(2ca) / \\\\surd{a^2 + (b + 2ct)^2}], a_t = [{2c(b + 2ct)} / \\\\surd{a^2 + (b + 2ct)^2}]\",\n \"F\": \"a_n = [(2ca) / \\\\surd{a^2 + (b + ct)^2}], a_t = [{c(b + 2ct)} / \\\\surd{a^2 + (b + ct)^2}]\",\n \"G\": \"a_n = [(2ca) / \\\\surd{a^2 + (b - 2ct)^2}], a_t = [{2c(b - 2ct)} / \\\\surd{a^2 + (b - 2ct)^2}]\",\n \"H\": \"a_n = [(2ca) / \\\\surd{a^2 + (b - 2ct)^2}], a_t = [{2c(b - 2ct)} / \\\\surd{a^2 - (b - 2ct)^2}]\",\n \"I\": \"a_n = [(2cb) / \\\\surd{a^2 + (b - 2ct)^2}], a_t = [{2c(a - 2ct)} / \\\\surd{a^2 + (b - 2ct)^2}]\",\n \"J\": \"a_n = [(2ca) / \\\\surd{a^2 - (b - 2ct)^2}], a_t = [{2c(b - 2ct)} / \\\\surd{a^2 - (b - 2ct)^2}]\"\n }\n}
input
question.json
{
"question": "Suppose a fluid particle moves according to the equations x = at, y =bt- ct^2, where a, b, c are constants. Find the normal and tangential accelerations.",
"options": {
"A": "a_n = [(2ca) / \\surd{a^2 + b^2}], a_t = [{2c(b - ct)} / \\surd{a^2 + b^2}]",
"B": "a_n = [(2cb) / \\surd{a^2 + (b + 2ct)^2}], a_t = [{2c(a + 2ct)} / \\surd{a^2 + (b + 2ct)^2}]",
"C": "a_n = [(2ca) / \\surd{(a - 2ct)^2 + b^2}], a_t = [{2c(b - ct)} / \\surd{(a - 2ct)^2 + b^2}]",
"D": "a_n = [c / \\surd{a^2 + (b - 2ct)^2}], a_t = [{2c(b - ct)} / \\surd{a^2 + (b - 2ct)^2}]",
"E": "a_n = [(2ca) / \\surd{a^2 + (b + 2ct)^2}], a_t = [{2c(b + 2ct)} / \\surd{a^2 + (b + 2ct)^2}]",
"F": "a_n = [(2ca) / \\surd{a^2 + (b + ct)^2}], a_t = [{c(b + 2ct)} / \\surd{a^2 + (b + ct)^2}]",
"G": "a_n = [(2ca) / \\surd{a^2 + (b - 2ct)^2}], a_t = [{2c(b - 2ct)} / \\surd{a^2 + (b - 2ct)^2}]",
"H": "a_n = [(2ca) / \\surd{a^2 + (b - 2ct)^2}], a_t = [{2c(b - 2ct)} / \\surd{a^2 - (b - 2ct)^2}]",
"I": "a_n = [(2cb) / \\surd{a^2 + (b - 2ct)^2}], a_t = [{2c(a - 2ct)} / \\surd{a^2 + (b - 2ct)^2}]",
"J": "a_n = [(2ca) / \\surd{a^2 - (b - 2ct)^2}], a_t = [{2c(b - 2ct)} / \\surd{a^2 - (b - 2ct)^2}]"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_004/ files.
Scored by a judge this pinned commit does not carry.
▸case_005Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Find the theoretical variance of the random variable with the following probability distribution. x Pr(X = x) 0 (1/4) 1 (1/2) 2 (1/8) 3 (1/8)\",\n \"options\": {\n \"A\": \"0.625\",\n \"B\": \"0.683\",\n \"C\": \"0.350\",\n \"D\": \"1.125\",\n \"E\": \"1.000\",\n \"F\": \"0.500\",\n \"G\": \"0.764\",\n \"H\": \"0.942\",\n \"I\": \"0.859\",\n \"J\": \"0.725\"\n }\n}
input
question.json
{
"question": "Find the theoretical variance of the random variable with the following probability distribution. x Pr(X = x) 0 (1/4) 1 (1/2) 2 (1/8) 3 (1/8)",
"options": {
"A": "0.625",
"B": "0.683",
"C": "0.350",
"D": "1.125",
"E": "1.000",
"F": "0.500",
"G": "0.764",
"H": "0.942",
"I": "0.859",
"J": "0.725"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_005/ files.
Scored by a judge this pinned commit does not carry.
▸case_006Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A flat plate 1.5 ft long and 5 ft wide is placed in a stream of water flowing at a velocity of 45 ft/s, such that the flow of water is parallel to the 1.5 ft. side of the plate. Calculate: a) the boundary layer thickness at the trailing edge. b) the drag force acting on the plate if the laminar boundary layer region ends at Re = 5 × 10^5. c) the drag force acting on the plate for turbulent smooth-wall flow from the leading edge. d) the drag force acting on the plate for turbulent rough-wall flow. The surface roughness of the plate is \\\\epsilon = 0.00035 ft.\",\n \"options\": {\n \"A\": \"a) 0.022 ft, b) 84 lb_f, c) 93 lb_f, d) 165 lb_f\",\n \"B\": \"a) 0.0257 ft, b) 89.56 lb_f, c) 97.1 lb_f, d) 173.7 lb_f\",\n \"C\": \"a) 0.028 ft, b) 92 lb_f, c) 102 lb_f, d) 178 lb_f\",\n \"D\": \"a) 0.03 ft, b) 88 lb_f, c) 95 lb_f, d) 170 lb_f\",\n \"E\": \"a) 0.0265 ft, b) 87 lb_f, c) 96 lb_f, d) 172 lb_f\",\n \"F\": \"a) 0.029 ft, b) 86 lb_f, c) 101 lb_f, d) 168 lb_f\",\n \"G\": \"a) 0.027 ft, b) 90 lb_f, c) 100 lb_f, d) 175 lb_f\",\n \"H\": \"a) 0.025 ft, b) 85 lb_f, c) 98 lb_f, d) 180 lb_f\",\n \"I\": \"a) 0.032 ft, b) 91 lb_f, c) 99 lb_f, d) 165 lb_f\",\n \"J\": \"a) 0.03 ft, b) 83 lb_f, c) 94 lb_f, d) 160 lb_f\"\n }\n}
input
question.json
{
"question": "A flat plate 1.5 ft long and 5 ft wide is placed in a stream of water flowing at a velocity of 45 ft/s, such that the flow of water is parallel to the 1.5 ft. side of the plate. Calculate: a) the boundary layer thickness at the trailing edge. b) the drag force acting on the plate if the laminar boundary layer region ends at Re = 5 × 10^5. c) the drag force acting on the plate for turbulent smooth-wall flow from the leading edge. d) the drag force acting on the plate for turbulent rough-wall flow. The surface roughness of the plate is \\epsilon = 0.00035 ft.",
"options": {
"A": "a) 0.022 ft, b) 84 lb_f, c) 93 lb_f, d) 165 lb_f",
"B": "a) 0.0257 ft, b) 89.56 lb_f, c) 97.1 lb_f, d) 173.7 lb_f",
"C": "a) 0.028 ft, b) 92 lb_f, c) 102 lb_f, d) 178 lb_f",
"D": "a) 0.03 ft, b) 88 lb_f, c) 95 lb_f, d) 170 lb_f",
"E": "a) 0.0265 ft, b) 87 lb_f, c) 96 lb_f, d) 172 lb_f",
"F": "a) 0.029 ft, b) 86 lb_f, c) 101 lb_f, d) 168 lb_f",
"G": "a) 0.027 ft, b) 90 lb_f, c) 100 lb_f, d) 175 lb_f",
"H": "a) 0.025 ft, b) 85 lb_f, c) 98 lb_f, d) 180 lb_f",
"I": "a) 0.032 ft, b) 91 lb_f, c) 99 lb_f, d) 165 lb_f",
"J": "a) 0.03 ft, b) 83 lb_f, c) 94 lb_f, d) 160 lb_f"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_006/ files.
Scored by a judge this pinned commit does not carry.
▸case_007Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Leopold Pospisil identifies four elements manifested by law: authority, universality, 'obligatio' - and fourthly?\",\n \"options\": {\n \"A\": \"Integrity.\",\n \"B\": \"Hierarchy.\",\n \"C\": \"Dispute.\",\n \"D\": \"Justice.\",\n \"E\": \"Harmony.\",\n \"F\": \"Regulation.\",\n \"G\": \"Punishment.\",\n \"H\": \"Sanction.\",\n \"I\": \"Order.\",\n \"J\": \"Equality.\"\n }\n}
input
question.json
{
"question": "Leopold Pospisil identifies four elements manifested by law: authority, universality, 'obligatio' - and fourthly?",
"options": {
"A": "Integrity.",
"B": "Hierarchy.",
"C": "Dispute.",
"D": "Justice.",
"E": "Harmony.",
"F": "Regulation.",
"G": "Punishment.",
"H": "Sanction.",
"I": "Order.",
"J": "Equality."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_007/ files.
Scored by a judge this pinned commit does not carry.
▸case_008Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"What is the impedance of a circuit if 100\\\\angle0° V causes a current7.07 + j7.07 A?\",\n \"options\": {\n \"A\": \"6.5 - j2.3 Ω\",\n \"B\": \"10 - j35 Ω\",\n \"C\": \"8.2 - j5.7 Ω\",\n \"D\": \"12.1 + j3.5 Ω\",\n \"E\": \"7.07 + j7.07 Ω\",\n \"F\": \"14 + j14 Ω\",\n \"G\": \"0 - j14 Ω\",\n \"H\": \"7.07 - j7.07 Ω\",\n \"I\": \"5 - j10 Ω\",\n \"J\": \"10 - j7 Ω\"\n }\n}
input
question.json
{
"question": "What is the impedance of a circuit if 100\\angle0° V causes a current7.07 + j7.07 A?",
"options": {
"A": "6.5 - j2.3 Ω",
"B": "10 - j35 Ω",
"C": "8.2 - j5.7 Ω",
"D": "12.1 + j3.5 Ω",
"E": "7.07 + j7.07 Ω",
"F": "14 + j14 Ω",
"G": "0 - j14 Ω",
"H": "7.07 - j7.07 Ω",
"I": "5 - j10 Ω",
"J": "10 - j7 Ω"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_008/ files.
Scored by a judge this pinned commit does not carry.
▸case_009Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A patent holder brought a patent infringement action in federal court against a licensee of the patent. The patent holder believed that a jury would be more sympathetic to his claims than a judge, and asked his lawyer to obtain a jury trial. What should the lawyer do to secure the patent holder's right to a jury trial?\",\n \"options\": {\n \"A\": \"File and serve a jury trial demand within 60 days after the close of the pleadings.\",\n \"B\": \"File and serve a jury trial demand at the close of discovery.\",\n \"C\": \"Submit a jury trial demand by email to the judge's office.\",\n \"D\": \"File and serve a jury trial demand within 30 days after the close of the pleadings.\",\n \"E\": \"File and serve a jury trial demand at the initial court hearing.\",\n \"F\": \"File and serve a complaint that excludes a jury trial demand.\",\n \"G\": \"Make a verbal request for a jury trial during the first court hearing.\",\n \"H\": \"Make a jury trial demand at the initial pretrial conference.\",\n \"I\": \"File and serve a complaint that includes a jury trial demand.\",\n \"J\": \"File and serve a jury trial demand within 14 days after the complaint was served.\"\n }\n}
input
question.json
{
"question": "A patent holder brought a patent infringement action in federal court against a licensee of the patent. The patent holder believed that a jury would be more sympathetic to his claims than a judge, and asked his lawyer to obtain a jury trial. What should the lawyer do to secure the patent holder's right to a jury trial?",
"options": {
"A": "File and serve a jury trial demand within 60 days after the close of the pleadings.",
"B": "File and serve a jury trial demand at the close of discovery.",
"C": "Submit a jury trial demand by email to the judge's office.",
"D": "File and serve a jury trial demand within 30 days after the close of the pleadings.",
"E": "File and serve a jury trial demand at the initial court hearing.",
"F": "File and serve a complaint that excludes a jury trial demand.",
"G": "Make a verbal request for a jury trial during the first court hearing.",
"H": "Make a jury trial demand at the initial pretrial conference.",
"I": "File and serve a complaint that includes a jury trial demand.",
"J": "File and serve a jury trial demand within 14 days after the complaint was served."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_009/ files.
Scored by a judge this pinned commit does not carry.
▸case_010Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"This jurisdiction has the following bribery statute in effect:\\\"Any person who offers or gives a thing of value to a government officeholder in exchange for official action is guilty of bribery. \\\"A real estate developer owned a large parcel of land in the suburbs. Although the developer wanted to build an office building on the property, the land was zoned residential. Due to the residential zoning, the developer could not pursue his planned development unless he received a variance from the building commission. The developer held a meeting with a member of the building commission to solicit his approval in securing a zoning variance. To do so, the developer gave the commission member $10,000 in exchange for his support in approving the zoning variance. Thereupon, the commission member voted to approve the variance, thus making it possible for the developer to commence construction of the office building. The developer was subsequently prosecuted for conspiracy to commit bribery. During the course of the trial, the commission member testified that he faked the agreement with the developer and would have approved the zoning variance regardless of whether the developer gave him any money. Furthermore, in his defense, the developer presented evidence that the other six members of the building commission voted affirmatively to approve the variance. If the jury believed that the commission member would have approved the variance even had he not received the $10,000, the developer should be found\",\n \"options\": {\n \"A\": \"guilty, because the commission member's agreement to accept the $10,000 was sufficient to form a conspiratorial objective.\",\n \"B\": \"guilty, because he gave the commission member the $10,000 in exchange for his approval of the zoning variance.\",\n \"C\": \"not guilty, because the commission member did not receive a thing of value, since he would have approved the variance regardless of receiving any payment from the developer.\",\n \"D\": \"not guilty, because there was no true agreement between the parties.\"\n }\n}
input
question.json
{
"question": "This jurisdiction has the following bribery statute in effect:\"Any person who offers or gives a thing of value to a government officeholder in exchange for official action is guilty of bribery. \"A real estate developer owned a large parcel of land in the suburbs. Although the developer wanted to build an office building on the property, the land was zoned residential. Due to the residential zoning, the developer could not pursue his planned development unless he received a variance from the building commission. The developer held a meeting with a member of the building commission to solicit his approval in securing a zoning variance. To do so, the developer gave the commission member $10,000 in exchange for his support in approving the zoning variance. Thereupon, the commission member voted to approve the variance, thus making it possible for the developer to commence construction of the office building. The developer was subsequently prosecuted for conspiracy to commit bribery. During the course of the trial, the commission member testified that he faked the agreement with the developer and would have approved the zoning variance regardless of whether the developer gave him any money. Furthermore, in his defense, the developer presented evidence that the other six members of the building commission voted affirmatively to approve the variance. If the jury believed that the commission member would have approved the variance even had he not received the $10,000, the developer should be found",
"options": {
"A": "guilty, because the commission member's agreement to accept the $10,000 was sufficient to form a conspiratorial objective.",
"B": "guilty, because he gave the commission member the $10,000 in exchange for his approval of the zoning variance.",
"C": "not guilty, because the commission member did not receive a thing of value, since he would have approved the variance regardless of receiving any payment from the developer.",
"D": "not guilty, because there was no true agreement between the parties."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_010/ files.
Scored by a judge this pinned commit does not carry.
▸case_011Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Buyer purchases an item from a merchant that turns out to be unfit, defective and unsafe for its ordinary purposes. Assuming no other warranties, which of the following can be invoked by the buyer?\",\n \"options\": {\n \"A\": \"Warranty of title\",\n \"B\": \"Implied warranty\",\n \"C\": \"Warranty of merchantability\",\n \"D\": \"Warranty of suitability\",\n \"E\": \"Warranty of non-infringement\",\n \"F\": \"Warranty of fitness for a particular purpose\",\n \"G\": \"Warranty of durability\",\n \"H\": \"Express warranty\",\n \"I\": \"Warranty of safety\",\n \"J\": \"Warranty of quality\"\n }\n}
input
question.json
{
"question": "Buyer purchases an item from a merchant that turns out to be unfit, defective and unsafe for its ordinary purposes. Assuming no other warranties, which of the following can be invoked by the buyer?",
"options": {
"A": "Warranty of title",
"B": "Implied warranty",
"C": "Warranty of merchantability",
"D": "Warranty of suitability",
"E": "Warranty of non-infringement",
"F": "Warranty of fitness for a particular purpose",
"G": "Warranty of durability",
"H": "Express warranty",
"I": "Warranty of safety",
"J": "Warranty of quality"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_011/ files.
Scored by a judge this pinned commit does not carry.
▸case_012Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A particle moves according to the equations x= sin 2t y=cos2t(1) z= e^-t wheret is the time. 1) Determine the velocity of the particle. 2) Determine the acceleration of the particle. 3) Find the magnitudes of the velocity and acceleration att = 0.\",\n \"options\": {\n \"A\": \"Velocity is (0, 2, -1), acceleration is (-4, -4, 0), velocity magnitude is √5, acceleration magnitude is √32\",\n \"B\": \"Velocity is (2, 0, -1), acceleration is (0, -4, 1), velocity magnitude is √5, acceleration magnitude is √17\",\n \"C\": \"Velocity is (1, 1, -1), acceleration is (0, -2, 1), velocity magnitude is √3, acceleration magnitude is √6\",\n \"D\": \"Velocity is (2, 2, -2), acceleration is (0, -4, 2), velocity magnitude is √12, acceleration magnitude is √20\",\n \"E\": \"Velocity is (2, 0, -1), acceleration is (0, -4, 1), velocity magnitude is √4, acceleration magnitude is √16\",\n \"F\": \"Velocity is (1, 0, -2), acceleration is (0, -3, 1), velocity magnitude is √5, acceleration magnitude is √17\",\n \"G\": \"Velocity is (2, -2, -1), acceleration is (4, 0, 1), velocity magnitude is √9, acceleration magnitude is √18\",\n \"H\": \"Velocity is (1, 2, -1), acceleration is (0, -3, 2), velocity magnitude is √4, acceleration magnitude is √16\",\n \"I\": \"Velocity is (2, -1, -2), acceleration is (4, 4, 1), velocity magnitude is √9, acceleration magnitude is √33\",\n \"J\": \"Velocity is (1, -2, 0), acceleration is (2, 3, -1), velocity magnitude is √6, acceleration magnitude is √14\"\n }\n}
input
question.json
{
"question": "A particle moves according to the equations x= sin 2t y=cos2t(1) z= e^-t wheret is the time. 1) Determine the velocity of the particle. 2) Determine the acceleration of the particle. 3) Find the magnitudes of the velocity and acceleration att = 0.",
"options": {
"A": "Velocity is (0, 2, -1), acceleration is (-4, -4, 0), velocity magnitude is √5, acceleration magnitude is √32",
"B": "Velocity is (2, 0, -1), acceleration is (0, -4, 1), velocity magnitude is √5, acceleration magnitude is √17",
"C": "Velocity is (1, 1, -1), acceleration is (0, -2, 1), velocity magnitude is √3, acceleration magnitude is √6",
"D": "Velocity is (2, 2, -2), acceleration is (0, -4, 2), velocity magnitude is √12, acceleration magnitude is √20",
"E": "Velocity is (2, 0, -1), acceleration is (0, -4, 1), velocity magnitude is √4, acceleration magnitude is √16",
"F": "Velocity is (1, 0, -2), acceleration is (0, -3, 1), velocity magnitude is √5, acceleration magnitude is √17",
"G": "Velocity is (2, -2, -1), acceleration is (4, 0, 1), velocity magnitude is √9, acceleration magnitude is √18",
"H": "Velocity is (1, 2, -1), acceleration is (0, -3, 2), velocity magnitude is √4, acceleration magnitude is √16",
"I": "Velocity is (2, -1, -2), acceleration is (4, 4, 1), velocity magnitude is √9, acceleration magnitude is √33",
"J": "Velocity is (1, -2, 0), acceleration is (2, 3, -1), velocity magnitude is √6, acceleration magnitude is √14"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_012/ files.
Scored by a judge this pinned commit does not carry.
▸case_013Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Find f_1(t) {_\\\\ast} f_2(t) if f_1(t) = 2e^-4tu(t) and f_2(t) = 5 cos 3t u(t).\",\n \"options\": {\n \"A\": \"f_1(t) {_\\u0007st} f_2(t) = (1.6 cos 3t + 1.2 sin 3t - 1.6e^-4t)u(t)\",\n \"B\": \"f_1(t) {_\\u0007st} f_2(t) = (1.2 cos 3t + 1.6 sin 3t - 1.3e^-4t)u(t)\",\n \"C\": \"f_1(t) {_\\u0007st} f_2(t) = (1.4 cos 3t + 1.2 sin 3t - 1.5e^-4t)u(t)\",\n \"D\": \"f_1(t) {_\\u0007st} f_2(t) = (1.6 cos 3t + 1.0 sin 3t - 1.6e^-4t)u(t)\"\n }\n}
input
question.json
{
"question": "Find f_1(t) {_\\ast} f_2(t) if f_1(t) = 2e^-4tu(t) and f_2(t) = 5 cos 3t u(t).",
"options": {
"A": "f_1(t) {_\u0007st} f_2(t) = (1.6 cos 3t + 1.2 sin 3t - 1.6e^-4t)u(t)",
"B": "f_1(t) {_\u0007st} f_2(t) = (1.2 cos 3t + 1.6 sin 3t - 1.3e^-4t)u(t)",
"C": "f_1(t) {_\u0007st} f_2(t) = (1.4 cos 3t + 1.2 sin 3t - 1.5e^-4t)u(t)",
"D": "f_1(t) {_\u0007st} f_2(t) = (1.6 cos 3t + 1.0 sin 3t - 1.6e^-4t)u(t)"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_013/ files.
Scored by a judge this pinned commit does not carry.
▸case_014Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Calculate the critical angle for an electromagnetic wave passingfrom the following dielectrics into air. Material\\\\epsilon/\\\\epsilon_0(Ratio of permittivity to that of air) Distilled water81.1 Glass9\",\n \"options\": {\n \"A\": \"9 degrees for distilled water, 81.1 degrees for glass\",\n \"B\": \"6.3 degrees for distilled water, 19.4 degrees for glass\",\n \"C\": \"81.1 degrees for distilled water, 9 degrees for glass\",\n \"D\": \"73 degrees for distilled water, 30 degrees for glass\",\n \"E\": \"20 degrees for distilled water, 6.3 degrees for glass\",\n \"F\": \"48.6 degrees for distilled water, 42.1 degrees for glass\",\n \"G\": \"25 degrees for distilled water, 89 degrees for glass\",\n \"H\": \"33 degrees for distilled water, 14 degrees for glass\",\n \"I\": \"13.2 degrees for distilled water, 25.8 degrees for glass\",\n \"J\": \"58 degrees for distilled water, 22 degrees for glass\"\n }\n}
input
question.json
{
"question": "Calculate the critical angle for an electromagnetic wave passingfrom the following dielectrics into air. Material\\epsilon/\\epsilon_0(Ratio of permittivity to that of air) Distilled water81.1 Glass9",
"options": {
"A": "9 degrees for distilled water, 81.1 degrees for glass",
"B": "6.3 degrees for distilled water, 19.4 degrees for glass",
"C": "81.1 degrees for distilled water, 9 degrees for glass",
"D": "73 degrees for distilled water, 30 degrees for glass",
"E": "20 degrees for distilled water, 6.3 degrees for glass",
"F": "48.6 degrees for distilled water, 42.1 degrees for glass",
"G": "25 degrees for distilled water, 89 degrees for glass",
"H": "33 degrees for distilled water, 14 degrees for glass",
"I": "13.2 degrees for distilled water, 25.8 degrees for glass",
"J": "58 degrees for distilled water, 22 degrees for glass"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_014/ files.
Scored by a judge this pinned commit does not carry.
▸case_015Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A state statute contained reasonable provisions for the regulation, sale, possession and use of certain weapons for self-defense in one's home, including semi-automatic rifles and handguns. The statute stressed registration and the individual's personal characteristics. The law prohibited the issuance of a license to felons and mentally ill persons or others presenting a danger of violence. It prohibited the commercial sale of guns from residential locations and required a commercial permit. Certain semi-automatic weapons were prohibited. A citizen who wished to purchase and maintain various guns in a manner violative of the statute sued the state. He claimed that the statute was a violation of the Second Amendment, which provides for the \\\"right to keep and bear arms.\\\" The case was accepted for review by the U.S. Supreme Court. What will the Court decide?\",\n \"options\": {\n \"A\": \"The statute is a reasonable regulation that will not hinder the right to possess a handgun for self-defense in the home and the individual's lawsuit will be dismissed.\",\n \"B\": \"The statute is unconstitutional because it violates the Second Amendment, thus the individual's lawsuit will be successful.\",\n \"C\": \"The statute is invalid as it excessively regulates the possession and use of firearms, thus the individual's lawsuit will be successful.\",\n \"D\": \"The statute is unreasonable as it infringes on the individual's right to bear arms, thus the man's lawsuit will be successful.\",\n \"E\": \"The Second Amendment only applies to militias, thus the individual's lawsuit will be dismissed.\",\n \"F\": \"Semi-automatic guns cannot be restricted in any way and the lawsuit will be successful.\",\n \"G\": \"The Second Amendment prohibits all forms of gun control because of the need for a strong militia, and the man's lawsuit will be successful.\",\n \"H\": \"The statute is a reasonable regulation but the individual's lawsuit will still be successful due to a technicality in the law.\",\n \"I\": \"The statute is constitutional as it aims to prevent the misuse of firearms, thus the individual's lawsuit will be dismissed.\",\n \"J\": \"The regulation of firearms for the safety of society is an absolute, unfettered right of the state governments and the individual's lawsuit is dismissed for that reason.\"\n }\n}
input
question.json
{
"question": "A state statute contained reasonable provisions for the regulation, sale, possession and use of certain weapons for self-defense in one's home, including semi-automatic rifles and handguns. The statute stressed registration and the individual's personal characteristics. The law prohibited the issuance of a license to felons and mentally ill persons or others presenting a danger of violence. It prohibited the commercial sale of guns from residential locations and required a commercial permit. Certain semi-automatic weapons were prohibited. A citizen who wished to purchase and maintain various guns in a manner violative of the statute sued the state. He claimed that the statute was a violation of the Second Amendment, which provides for the \"right to keep and bear arms.\" The case was accepted for review by the U.S. Supreme Court. What will the Court decide?",
"options": {
"A": "The statute is a reasonable regulation that will not hinder the right to possess a handgun for self-defense in the home and the individual's lawsuit will be dismissed.",
"B": "The statute is unconstitutional because it violates the Second Amendment, thus the individual's lawsuit will be successful.",
"C": "The statute is invalid as it excessively regulates the possession and use of firearms, thus the individual's lawsuit will be successful.",
"D": "The statute is unreasonable as it infringes on the individual's right to bear arms, thus the man's lawsuit will be successful.",
"E": "The Second Amendment only applies to militias, thus the individual's lawsuit will be dismissed.",
"F": "Semi-automatic guns cannot be restricted in any way and the lawsuit will be successful.",
"G": "The Second Amendment prohibits all forms of gun control because of the need for a strong militia, and the man's lawsuit will be successful.",
"H": "The statute is a reasonable regulation but the individual's lawsuit will still be successful due to a technicality in the law.",
"I": "The statute is constitutional as it aims to prevent the misuse of firearms, thus the individual's lawsuit will be dismissed.",
"J": "The regulation of firearms for the safety of society is an absolute, unfettered right of the state governments and the individual's lawsuit is dismissed for that reason."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_015/ files.
Scored by a judge this pinned commit does not carry.
▸case_016Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A factory draws a lagging load of 2000 kW at a power factor of 0.6 from 6000 V mains. A synchronous capacitor is purchased to raise the overall power factor to unity. Assuming that the synchronous capacitor losses are 275 kW, calculate: (a) Originalkilovarsof lagging load. (b)kvarsof correction needed to bring the power factor to unity. (c)kVArating of synchronous capacitor and its power factor.\",\n \"options\": {\n \"A\": \"2200 kvars, 2400 kVA, 0.15 leading\",\n \"B\": \"2500 kvars, 3000 kVA, 0.1 leading\",\n \"C\": \"2667 kvars, 3300 kVA, 0.09 leading\",\n \"D\": \"1800 kvars, 2100 kVA, 0.12 leading\",\n \"E\": \"3000kvars, 2500kVA, 0.2 leading\",\n \"F\": \"2000kvars, 2300kVA, 0.15 leading\",\n \"G\": \"3500kvars, 2900kVA, 0.05 leading\",\n \"H\": \"1600 kvars, 2000 kVA, 0.2 leading\",\n \"I\": \"2800 kvars, 3100 kVA, 0.08 leading\",\n \"J\": \"2667kvars, 2755kVA, 0.103 leading\"\n }\n}
input
question.json
{
"question": "A factory draws a lagging load of 2000 kW at a power factor of 0.6 from 6000 V mains. A synchronous capacitor is purchased to raise the overall power factor to unity. Assuming that the synchronous capacitor losses are 275 kW, calculate: (a) Originalkilovarsof lagging load. (b)kvarsof correction needed to bring the power factor to unity. (c)kVArating of synchronous capacitor and its power factor.",
"options": {
"A": "2200 kvars, 2400 kVA, 0.15 leading",
"B": "2500 kvars, 3000 kVA, 0.1 leading",
"C": "2667 kvars, 3300 kVA, 0.09 leading",
"D": "1800 kvars, 2100 kVA, 0.12 leading",
"E": "3000kvars, 2500kVA, 0.2 leading",
"F": "2000kvars, 2300kVA, 0.15 leading",
"G": "3500kvars, 2900kVA, 0.05 leading",
"H": "1600 kvars, 2000 kVA, 0.2 leading",
"I": "2800 kvars, 3100 kVA, 0.08 leading",
"J": "2667kvars, 2755kVA, 0.103 leading"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_016/ files.
Scored by a judge this pinned commit does not carry.
▸case_017Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A fortune teller told fortunes by means of Tarot cards. An elderly woman, who was worried about her failing health, had heard that the fortuneteller was clairvoyant and could see into the future. Consequently, the woman decided to see the fortuneteller in order to have her Tarot cards read. As the fortuneteller was telling the woman her fortune, she suddenly said, \\\"I have a vision. If you give me $25,000 tomorrow, you will live to be 100 years old. \\\" The woman, who was 72 years of age, believed the fortuneteller and gave her the money the next day. The following week, the woman's physician informed her that she had a serious heart disease and he didn't expect her to live for more than a year or two. If the woman asserts a claim against the fortuneteller based on deceit, the plaintiff should\",\n \"options\": {\n \"A\": \"prevail, because she relied to her detriment on the fortune teller's foretelling.\",\n \"B\": \"prevail, if the fortuneteller did not honestly believe that the woman would live to be 100 years of age.\",\n \"C\": \"not prevail, unless there was a fiduciary relationship between the parties.\",\n \"D\": \"not prevail, unless the fortuneteller warranted the truth as believed.\"\n }\n}
input
question.json
{
"question": "A fortune teller told fortunes by means of Tarot cards. An elderly woman, who was worried about her failing health, had heard that the fortuneteller was clairvoyant and could see into the future. Consequently, the woman decided to see the fortuneteller in order to have her Tarot cards read. As the fortuneteller was telling the woman her fortune, she suddenly said, \"I have a vision. If you give me $25,000 tomorrow, you will live to be 100 years old. \" The woman, who was 72 years of age, believed the fortuneteller and gave her the money the next day. The following week, the woman's physician informed her that she had a serious heart disease and he didn't expect her to live for more than a year or two. If the woman asserts a claim against the fortuneteller based on deceit, the plaintiff should",
"options": {
"A": "prevail, because she relied to her detriment on the fortune teller's foretelling.",
"B": "prevail, if the fortuneteller did not honestly believe that the woman would live to be 100 years of age.",
"C": "not prevail, unless there was a fiduciary relationship between the parties.",
"D": "not prevail, unless the fortuneteller warranted the truth as believed."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_017/ files.
Scored by a judge this pinned commit does not carry.
▸case_018Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"An ideal gas is enclosed inside a chamber with a volume of 0.1 ft^3 at 115°C, 690kPa. It then expandsisentropicallyto a final pressure of 138kPa. Calculate the work done during the process, assuming that for this gas c_v= 0.7201 kJ/kg-°K c_p = 1.0048 kJ/kg-°K\",\n \"options\": {\n \"A\": \"2.3456 kJ\",\n \"B\": \"3.2109 kJ\",\n \"C\": \"2.0000 kJ\",\n \"D\": \"1.5678 kJ\",\n \"E\": \"1.4567 kJ\",\n \"F\": \"2.6789 kJ\",\n \"G\": \"2.1234 kJ\",\n \"H\": \"1.8122 kJ\",\n \"I\": \"1.2345 kJ\",\n \"J\": \"0.9876 kJ\"\n }\n}
input
question.json
{
"question": "An ideal gas is enclosed inside a chamber with a volume of 0.1 ft^3 at 115°C, 690kPa. It then expandsisentropicallyto a final pressure of 138kPa. Calculate the work done during the process, assuming that for this gas c_v= 0.7201 kJ/kg-°K c_p = 1.0048 kJ/kg-°K",
"options": {
"A": "2.3456 kJ",
"B": "3.2109 kJ",
"C": "2.0000 kJ",
"D": "1.5678 kJ",
"E": "1.4567 kJ",
"F": "2.6789 kJ",
"G": "2.1234 kJ",
"H": "1.8122 kJ",
"I": "1.2345 kJ",
"J": "0.9876 kJ"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_018/ files.
Scored by a judge this pinned commit does not carry.
▸case_019Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"How much capacitance is required to resonate a coil of 0.001mHat a frequency of 2MHz?\",\n \"options\": {\n \"A\": \"5,000pF\",\n \"B\": \"7,000pF\",\n \"C\": \"9,500pF\",\n \"D\": \"8,000pF\",\n \"E\": \"5,500pF\",\n \"F\": \"6,330pF\",\n \"G\": \"7,250pF\",\n \"H\": \"6,500pF\",\n \"I\": \"10,000pF\",\n \"J\": \"4,750pF\"\n }\n}
input
question.json
{
"question": "How much capacitance is required to resonate a coil of 0.001mHat a frequency of 2MHz?",
"options": {
"A": "5,000pF",
"B": "7,000pF",
"C": "9,500pF",
"D": "8,000pF",
"E": "5,500pF",
"F": "6,330pF",
"G": "7,250pF",
"H": "6,500pF",
"I": "10,000pF",
"J": "4,750pF"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_019/ files.
Scored by a judge this pinned commit does not carry.
▸case_020Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"The smallest change in measured variable to which instrument will respond is\",\n \"options\": {\n \"A\": \"calibration.\",\n \"B\": \"sensitivity.\",\n \"C\": \"resolution.\",\n \"D\": \"repeatability.\",\n \"E\": \"linearity.\",\n \"F\": \"precision.\",\n \"G\": \"accuracy.\",\n \"H\": \"scale.\",\n \"I\": \"range.\",\n \"J\": \"reliability.\"\n }\n}
input
question.json
{
"question": "The smallest change in measured variable to which instrument will respond is",
"options": {
"A": "calibration.",
"B": "sensitivity.",
"C": "resolution.",
"D": "repeatability.",
"E": "linearity.",
"F": "precision.",
"G": "accuracy.",
"H": "scale.",
"I": "range.",
"J": "reliability."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_020/ files.
Scored by a judge this pinned commit does not carry.
▸case_021Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"In 1993, a rancher had good record title to a 20-acre orange grove in fee simple absolute. In 1994, the rancher delivered to his son, for a sum of $1,000, a deed signed by the rancher, naming the son and his heirs as grantee, and appearing valid on its face. The son neglected to record the deed. In 1998, a farmer, aware of the existence of the rancher-to-son deed, sought out the rancher and asked to buy for $10,000 a deed to the orange grove from the rancher to the fanner and his heirs. The rancher executed such a deed, and the fanner promptly recorded it. The farmer's intent was to acquire color of title and obtain ownership of the orange grove by adverse possession. In 1998, the farmer constructed a fence around the orange grove. In 1999, the son presented his deed of the orange grove to a retiree, and for $15,000, paid by the retiree, signed and delivered a deed of the orange grove in favor of the retiree and his heirs. After receiving the deed, the retiree made no effort to search the title, to examine the property, or to record the deed. In 2003, a buyer paid the fanner $20,000, and the farmer delivered to the buyer a deed of the orange grove in favor of the buyer and his heirs. The buyer had examined the property, had searched the title, and had no knowledge of the farmer's awareness of the prior rancher-to-son instrument. Although the buyer did not reside on the property, he regularly visited the orange grove twice a week. The buyer recorded his deed. In 2007 for $25,000 paid by an orange grower, the retiree signed and delivered adeed of the orange grove naming the orange grower and his heirs as grantees. Before the grower had paid the retiree and taken his deed, the grower visited the orange grove and observed the fence. However, the buyer was not present when the grower visited the property and nothing suggested who \u0080\u0094if anyone \u0080\u0094was using it. In any case, the grower did not attempt to search the title before making his purchase. This jurisdiction uses Grantor \u0080\u0094Grantee Indices and has no Tract Index. In 2008, what is the present state of title to the orange grove if the jurisdiction's recording act provides: \\\"Every conveyance of real property shall be invalid as against any person, except the grantor, his heirs, and assigns, having actual notice thereof, unless it is recorded as provided by statute\\\"?\",\n \"options\": {\n \"A\": \"In a notice jurisdiction, the grower, as a subsequent bonafide purchaser, is only chargeable with notice of what appears in his chain of title and, therefore, would acquire record title to the orange grove.\",\n \"B\": \"In a race \u0080\u0094notice jurisdiction, the farmer would acquire equitable title to the orange grove, because he erected the fence and failed to inform the buyer of the prior rancher-to-son instrument.\",\n \"C\": \"In a race-notice jurisdiction, the farmer, having recorded his deed first, would acquire record title to the orange grove.\",\n \"D\": \"In a race-notice jurisdiction, the grower, being the last to record, would acquire record title to the orange grove.\",\n \"E\": \"In a notice jurisdiction, the retiree would acquire record title to the orange grove even though he failed to record his deed.\",\n \"F\": \"In a race-notice jurisdiction, the son would acquire record title to the orange grove, because he is the original grantee.\",\n \"G\": \"In a race jurisdiction, the buyer, having recorded his deed before the grower, would acquire record title to the orange grove.\",\n \"H\": \"In a notice jurisdiction, the retiree, despite not recording his deed, would acquire equitable title to the orange grove due to his payment and receipt of the deed.\",\n \"I\": \"In a notice jurisdiction, the rancher retains record title to the orange grove, as he never properly conveyed his title.\"\n }\n}
input
question.json
{
"question": "In 1993, a rancher had good record title to a 20-acre orange grove in fee simple absolute. In 1994, the rancher delivered to his son, for a sum of $1,000, a deed signed by the rancher, naming the son and his heirs as grantee, and appearing valid on its face. The son neglected to record the deed. In 1998, a farmer, aware of the existence of the rancher-to-son deed, sought out the rancher and asked to buy for $10,000 a deed to the orange grove from the rancher to the fanner and his heirs. The rancher executed such a deed, and the fanner promptly recorded it. The farmer's intent was to acquire color of title and obtain ownership of the orange grove by adverse possession. In 1998, the farmer constructed a fence around the orange grove. In 1999, the son presented his deed of the orange grove to a retiree, and for $15,000, paid by the retiree, signed and delivered a deed of the orange grove in favor of the retiree and his heirs. After receiving the deed, the retiree made no effort to search the title, to examine the property, or to record the deed. In 2003, a buyer paid the fanner $20,000, and the farmer delivered to the buyer a deed of the orange grove in favor of the buyer and his heirs. The buyer had examined the property, had searched the title, and had no knowledge of the farmer's awareness of the prior rancher-to-son instrument. Although the buyer did not reside on the property, he regularly visited the orange grove twice a week. The buyer recorded his deed. In 2007 for $25,000 paid by an orange grower, the retiree signed and delivered adeed of the orange grove naming the orange grower and his heirs as grantees. Before the grower had paid the retiree and taken his deed, the grower visited the orange grove and observed the fence. However, the buyer was not present when the grower visited the property and nothing suggested who if anyone was using it. In any case, the grower did not attempt to search the title before making his purchase. This jurisdiction uses Grantor Grantee Indices and has no Tract Index. In 2008, what is the present state of title to the orange grove if the jurisdiction's recording act provides: \"Every conveyance of real property shall be invalid as against any person, except the grantor, his heirs, and assigns, having actual notice thereof, unless it is recorded as provided by statute\"?",
"options": {
"A": "In a notice jurisdiction, the grower, as a subsequent bonafide purchaser, is only chargeable with notice of what appears in his chain of title and, therefore, would acquire record title to the orange grove.",
"B": "In a race notice jurisdiction, the farmer would acquire equitable title to the orange grove, because he erected the fence and failed to inform the buyer of the prior rancher-to-son instrument.",
"C": "In a race-notice jurisdiction, the farmer, having recorded his deed first, would acquire record title to the orange grove.",
"D": "In a race-notice jurisdiction, the grower, being the last to record, would acquire record title to the orange grove.",
"E": "In a notice jurisdiction, the retiree would acquire record title to the orange grove even though he failed to record his deed.",
"F": "In a race-notice jurisdiction, the son would acquire record title to the orange grove, because he is the original grantee.",
"G": "In a race jurisdiction, the buyer, having recorded his deed before the grower, would acquire record title to the orange grove.",
"H": "In a notice jurisdiction, the retiree, despite not recording his deed, would acquire equitable title to the orange grove due to his payment and receipt of the deed.",
"I": "In a notice jurisdiction, the rancher retains record title to the orange grove, as he never properly conveyed his title."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_021/ files.
Scored by a judge this pinned commit does not carry.
▸case_022Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"James makes a written offer to Fred for the sale of land for $100,000. In this offer, James states, \\\"This offer will not be revocable for a ten-day period of time.\\\"\u009d The offer was signed on April 1, mailed to Fred on April 3 and received by Fred on April 5th. Which of the following is true?\",\n \"options\": {\n \"A\": \"Fred's acceptance on April 5 will not create a contract if James dies on April 4.\",\n \"B\": \"Fred's unqualified acceptance on April 10 will not create a contract.\",\n \"C\": \"Fred stating on April 5, \\\"I accept your offer, but will pay only $90,000\\\" creates a contract.\",\n \"D\": \"Fred's unqualified acceptance on April 11 will create a contract.\",\n \"E\": \"Fred stating on April 6, \\\"I accept your offer, but will pay in 60 days\\\" creates a contract.\",\n \"F\": \"James revoking the offer on April 4 will not invalidate the contract.\"\n }\n}
input
question.json
{
"question": "James makes a written offer to Fred for the sale of land for $100,000. In this offer, James states, \"This offer will not be revocable for a ten-day period of time.\" The offer was signed on April 1, mailed to Fred on April 3 and received by Fred on April 5th. Which of the following is true?",
"options": {
"A": "Fred's acceptance on April 5 will not create a contract if James dies on April 4.",
"B": "Fred's unqualified acceptance on April 10 will not create a contract.",
"C": "Fred stating on April 5, \"I accept your offer, but will pay only $90,000\" creates a contract.",
"D": "Fred's unqualified acceptance on April 11 will create a contract.",
"E": "Fred stating on April 6, \"I accept your offer, but will pay in 60 days\" creates a contract.",
"F": "James revoking the offer on April 4 will not invalidate the contract."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_022/ files.
Scored by a judge this pinned commit does not carry.
▸case_023Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"The electric displacementDin a given spherical region is given as D= {(5r^2) / 4} r̂ (C/m^2) Determine the total charge enclosed by the volume r = 4m, and \\\\texttheta = \\\\pi/4.\",\n \"options\": {\n \"A\": \"589.1 c\",\n \"B\": \"450.5 c\",\n \"C\": \"900.3 c\",\n \"D\": \"680.4 c\",\n \"E\": \"550.8 c\",\n \"F\": \"735.2 c\",\n \"G\": \"812.7 c\",\n \"H\": \"320.3 c\",\n \"I\": \"400.0 c\",\n \"J\": \"1001.2 c\"\n }\n}
input
question.json
{
"question": "The electric displacementDin a given spherical region is given as D= {(5r^2) / 4} r̂ (C/m^2) Determine the total charge enclosed by the volume r = 4m, and \\texttheta = \\pi/4.",
"options": {
"A": "589.1 c",
"B": "450.5 c",
"C": "900.3 c",
"D": "680.4 c",
"E": "550.8 c",
"F": "735.2 c",
"G": "812.7 c",
"H": "320.3 c",
"I": "400.0 c",
"J": "1001.2 c"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_023/ files.
Scored by a judge this pinned commit does not carry.
▸case_024Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"The annular spacebetween two concentricaluminium spheresis evacuated to provide insulation to the system. The radii of the inner and outer spheres are 0.75 ft and 1.0 ft respectively. The inner sphere contains liquefied oxygen and theouter sphere is maintained at 45°F. The boiling temperatureof oxygen is - 297°F and the emissivity of aluminiumis \\\\epsilon = 0.03. Determine the rate of heat flow to the oxygenby radiation.\",\n \"options\": {\n \"A\": \"12.34 Btu/hr\",\n \"B\": \"13.89 Btu/hr\",\n \"C\": \"18.45 Btu/hr\",\n \"D\": \"10.25 Btu/hr\",\n \"E\": \"20.36 Btu/hr\",\n \"F\": \"15.26 Btu/hr\",\n \"G\": \"25.10 Btu/hr\",\n \"H\": \"17.02 Btu/hr\",\n \"I\": \"8.97 Btu/hr\",\n \"J\": \"22.58 Btu/hr\"\n }\n}
input
question.json
{
"question": "The annular spacebetween two concentricaluminium spheresis evacuated to provide insulation to the system. The radii of the inner and outer spheres are 0.75 ft and 1.0 ft respectively. The inner sphere contains liquefied oxygen and theouter sphere is maintained at 45°F. The boiling temperatureof oxygen is - 297°F and the emissivity of aluminiumis \\epsilon = 0.03. Determine the rate of heat flow to the oxygenby radiation.",
"options": {
"A": "12.34 Btu/hr",
"B": "13.89 Btu/hr",
"C": "18.45 Btu/hr",
"D": "10.25 Btu/hr",
"E": "20.36 Btu/hr",
"F": "15.26 Btu/hr",
"G": "25.10 Btu/hr",
"H": "17.02 Btu/hr",
"I": "8.97 Btu/hr",
"J": "22.58 Btu/hr"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_024/ files.
Scored by a judge this pinned commit does not carry.
▸case_025Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Evaluate the force on the spring of a cone-clutch delivering a torqueof 2000 lbs.-in. The mean diameter for the disc is 18 in., the clutch angle is 10° and the coefficient of friction is 0.30.\",\n \"options\": {\n \"A\": \"100 lbs\",\n \"B\": \"175 lbs\",\n \"C\": \"128.6 lbs\",\n \"D\": \"110.5 lbs\",\n \"E\": \"85 lbs\",\n \"F\": \"150 lbs\",\n \"G\": \"200 lbs\",\n \"H\": \"95 lbs\",\n \"I\": \"215 lbs\",\n \"J\": \"250 lbs\"\n }\n}
input
question.json
{
"question": "Evaluate the force on the spring of a cone-clutch delivering a torqueof 2000 lbs.-in. The mean diameter for the disc is 18 in., the clutch angle is 10° and the coefficient of friction is 0.30.",
"options": {
"A": "100 lbs",
"B": "175 lbs",
"C": "128.6 lbs",
"D": "110.5 lbs",
"E": "85 lbs",
"F": "150 lbs",
"G": "200 lbs",
"H": "95 lbs",
"I": "215 lbs",
"J": "250 lbs"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_025/ files.
Scored by a judge this pinned commit does not carry.
▸case_026Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A national association brought suit in federal court against a city. The suit charged the city with violating several federal statutes. During the trial, the association's attorney called the city attorney as an adverse witness. The city attorney asserted the attorney-client privilege and objected that the matters communicated to him were intended to be confidential. The issues raised by the objections of the city attorney should be resolved under\",\n \"options\": {\n \"A\": \"the privilege law of the state where the national association is located.\",\n \"B\": \"the privilege law of the state where the city is located.\",\n \"C\": \"either federal privilege law or the privilege law of the forum state as determined by the court.\",\n \"D\": \"city’s privilege law.\",\n \"E\": \"either federal privilege law or the privilege law of the city as determined by the court.\",\n \"F\": \"either federal privilege law or the privilege law of the forum state, whichever the court determines is more likely to admit the evidence.\",\n \"G\": \"federal privilege law.\",\n \"H\": \"federal privilege law or international privilege law, as determined by the court.\",\n \"I\": \"the privilege law of the forum state.\",\n \"J\": \"international privilege law.\"\n }\n}
input
question.json
{
"question": "A national association brought suit in federal court against a city. The suit charged the city with violating several federal statutes. During the trial, the association's attorney called the city attorney as an adverse witness. The city attorney asserted the attorney-client privilege and objected that the matters communicated to him were intended to be confidential. The issues raised by the objections of the city attorney should be resolved under",
"options": {
"A": "the privilege law of the state where the national association is located.",
"B": "the privilege law of the state where the city is located.",
"C": "either federal privilege law or the privilege law of the forum state as determined by the court.",
"D": "city’s privilege law.",
"E": "either federal privilege law or the privilege law of the city as determined by the court.",
"F": "either federal privilege law or the privilege law of the forum state, whichever the court determines is more likely to admit the evidence.",
"G": "federal privilege law.",
"H": "federal privilege law or international privilege law, as determined by the court.",
"I": "the privilege law of the forum state.",
"J": "international privilege law."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_026/ files.
Scored by a judge this pinned commit does not carry.
▸case_027Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Electrical analogous quantity for dash-pot in force-current analogy is\",\n \"options\": {\n \"A\": \"reactance.\",\n \"B\": \"admittance.\",\n \"C\": \"reluctance.\",\n \"D\": \"transconductance.\",\n \"E\": \"susceptance.\",\n \"F\": \"capacitance.\",\n \"G\": \"impedance.\",\n \"H\": \"resistance.\",\n \"I\": \"inductance.\",\n \"J\": \"conductance.\"\n }\n}
input
question.json
{
"question": "Electrical analogous quantity for dash-pot in force-current analogy is",
"options": {
"A": "reactance.",
"B": "admittance.",
"C": "reluctance.",
"D": "transconductance.",
"E": "susceptance.",
"F": "capacitance.",
"G": "impedance.",
"H": "resistance.",
"I": "inductance.",
"J": "conductance."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_027/ files.
Scored by a judge this pinned commit does not carry.
▸case_028Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A wet bulb thermometer is showing15.55°C when dry airat atmosphericpressure is blowing across the thermometer. Estimate the dry air temperature.\",\n \"options\": {\n \"A\": \"15.55°C\",\n \"B\": \"55.85°C\",\n \"C\": \"18.00°C\",\n \"D\": \"45.75°C\",\n \"E\": \"25.00°C\"\n }\n}
input
question.json
{
"question": "A wet bulb thermometer is showing15.55°C when dry airat atmosphericpressure is blowing across the thermometer. Estimate the dry air temperature.",
"options": {
"A": "15.55°C",
"B": "55.85°C",
"C": "18.00°C",
"D": "45.75°C",
"E": "25.00°C"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_028/ files.
Scored by a judge this pinned commit does not carry.
▸case_029Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A neighborhood homeowners' association hired a security company to provide an armed guard to patrol the neighborhood. One evening, the guard saw a young man stealing a rake that a homeowner in the neighborhood had left outside in his yard. The guard ran toward the young man. Seeing the guard, the young man dropped the rake and began to flee. Believing that the young man was about to escape, the guard pulled out his gun and fired a shot at the young man. The bullet struck the young man in the arm. If the young man asserts a claim against the security company for damages for his injuries, the young man will\",\n \"options\": {\n \"A\": \"prevail, because the guard did not give a warning that he had a gun and was about to shoot.\",\n \"B\": \"not prevail, because the homeowner's association approved the use of force in such situations.\",\n \"C\": \"prevail, because the guard should have called the police instead of taking matters into his own hands.\",\n \"D\": \"not prevail, because the guard acted within his rights to protect property within the neighborhood.\",\n \"E\": \"prevail, because the guard used unreasonable force to protect the homeowner's property.\",\n \"F\": \"not prevail, because the young man was engaged in theft when he was shot.\",\n \"G\": \"not prevail, because the injury was a result of the young man's own criminal activity.\",\n \"H\": \"not prevail, because the young man was trespassing on the homeowner's property.\"\n }\n}
input
question.json
{
"question": "A neighborhood homeowners' association hired a security company to provide an armed guard to patrol the neighborhood. One evening, the guard saw a young man stealing a rake that a homeowner in the neighborhood had left outside in his yard. The guard ran toward the young man. Seeing the guard, the young man dropped the rake and began to flee. Believing that the young man was about to escape, the guard pulled out his gun and fired a shot at the young man. The bullet struck the young man in the arm. If the young man asserts a claim against the security company for damages for his injuries, the young man will",
"options": {
"A": "prevail, because the guard did not give a warning that he had a gun and was about to shoot.",
"B": "not prevail, because the homeowner's association approved the use of force in such situations.",
"C": "prevail, because the guard should have called the police instead of taking matters into his own hands.",
"D": "not prevail, because the guard acted within his rights to protect property within the neighborhood.",
"E": "prevail, because the guard used unreasonable force to protect the homeowner's property.",
"F": "not prevail, because the young man was engaged in theft when he was shot.",
"G": "not prevail, because the injury was a result of the young man's own criminal activity.",
"H": "not prevail, because the young man was trespassing on the homeowner's property."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_029/ files.
Scored by a judge this pinned commit does not carry.
▸case_030Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"What is an act jure gestionis?\",\n \"options\": {\n \"A\": \"Acts jure gestionis are all actions performed by the State in a public capacity\",\n \"B\": \"All conduct undertaken by government officials in their free time is considered jus gestionis.\",\n \"C\": \"Acts jure gestionis are those undertaken by States in a private capacity\",\n \"D\": \"All international treaties signed by the State are acts jure gestionis\",\n \"E\": \"Acts jure gestionis are all decisions made by State courts\",\n \"F\": \"Acts jure gestionis are all legislative decisions made by the State.\",\n \"G\": \"All purchases by the State are acts jure gestionis\",\n \"H\": \"All diplomatic missions undertaken by the State are acts jure gestionis\",\n \"I\": \"Acts jure gestionis are military actions undertaken by States\",\n \"J\": \"All acts undertaken by State corporations are jure gestionis\"\n }\n}
input
question.json
{
"question": "What is an act jure gestionis?",
"options": {
"A": "Acts jure gestionis are all actions performed by the State in a public capacity",
"B": "All conduct undertaken by government officials in their free time is considered jus gestionis.",
"C": "Acts jure gestionis are those undertaken by States in a private capacity",
"D": "All international treaties signed by the State are acts jure gestionis",
"E": "Acts jure gestionis are all decisions made by State courts",
"F": "Acts jure gestionis are all legislative decisions made by the State.",
"G": "All purchases by the State are acts jure gestionis",
"H": "All diplomatic missions undertaken by the State are acts jure gestionis",
"I": "Acts jure gestionis are military actions undertaken by States",
"J": "All acts undertaken by State corporations are jure gestionis"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_030/ files.
Scored by a judge this pinned commit does not carry.
▸case_031Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A state government enacted a law that prohibited the state's agencies from purchasing services or goods from and companies doing business with Cuba. The federal government at the time was implementing a thaw in relations with the Cuban government. The U.S. Congress passed a statute regarding various policies and dealings toward Cuba. A National Trade Council sued the state in federal court seeking an injunction and declaratory judgment that the state law toward Cuba was unconstitutional. 30 of the business entities that were discriminated against by the state were members of the council. Is the federal court likely to grant the declaratory judgment and declare the state law unconstitutional?\",\n \"options\": {\n \"A\": \"Yes, the state law is unconstitutional because it violates the Supremacy Clause of the Constitution.\",\n \"B\": \"Yes, the state law is unconstitutional because it illegally interfered with the foreign affairs power of the federal government.\",\n \"C\": \"Yes, the state law is unconstitutional because it denies equal protection to citizens of other states.\",\n \"D\": \"No, the state law is constitutional because it aligns with the federal government's previous policy on Cuba.\",\n \"E\": \"No, the state law is constitutional as it is within the state's power to regulate commerce within its borders.\",\n \"F\": \"Yes, the state law is unconstitutional because it interferes with the President's exclusive power to recognize foreign governments.\",\n \"G\": \"No, a state has the sovereign right to pass its own laws concerning its policies toward foreign countries.\",\n \"H\": \"No, the state law is constitutional because it does not interfere with any federally protected rights.\",\n \"I\": \"Yes, the state law is unconstitutional because it constitutes a form of economic discrimination.\",\n \"J\": \"No, the state law was consistent with the federal law and with federal policy on Cuba.\"\n }\n}
input
question.json
{
"question": "A state government enacted a law that prohibited the state's agencies from purchasing services or goods from and companies doing business with Cuba. The federal government at the time was implementing a thaw in relations with the Cuban government. The U.S. Congress passed a statute regarding various policies and dealings toward Cuba. A National Trade Council sued the state in federal court seeking an injunction and declaratory judgment that the state law toward Cuba was unconstitutional. 30 of the business entities that were discriminated against by the state were members of the council. Is the federal court likely to grant the declaratory judgment and declare the state law unconstitutional?",
"options": {
"A": "Yes, the state law is unconstitutional because it violates the Supremacy Clause of the Constitution.",
"B": "Yes, the state law is unconstitutional because it illegally interfered with the foreign affairs power of the federal government.",
"C": "Yes, the state law is unconstitutional because it denies equal protection to citizens of other states.",
"D": "No, the state law is constitutional because it aligns with the federal government's previous policy on Cuba.",
"E": "No, the state law is constitutional as it is within the state's power to regulate commerce within its borders.",
"F": "Yes, the state law is unconstitutional because it interferes with the President's exclusive power to recognize foreign governments.",
"G": "No, a state has the sovereign right to pass its own laws concerning its policies toward foreign countries.",
"H": "No, the state law is constitutional because it does not interfere with any federally protected rights.",
"I": "Yes, the state law is unconstitutional because it constitutes a form of economic discrimination.",
"J": "No, the state law was consistent with the federal law and with federal policy on Cuba."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_031/ files.
Scored by a judge this pinned commit does not carry.
▸case_032Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"How many bits are required to store one BCD digit ?\",\n \"options\": {\n \"A\": \"7\",\n \"B\": \"10\",\n \"C\": \"2\",\n \"D\": \"3\",\n \"E\": \"5\",\n \"F\": \"4\",\n \"G\": \"9\",\n \"H\": \"6\",\n \"I\": \"1\",\n \"J\": \"8\"\n }\n}
input
question.json
{
"question": "How many bits are required to store one BCD digit ?",
"options": {
"A": "7",
"B": "10",
"C": "2",
"D": "3",
"E": "5",
"F": "4",
"G": "9",
"H": "6",
"I": "1",
"J": "8"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_032/ files.
Scored by a judge this pinned commit does not carry.
▸case_033Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A young man suggested to his friend that they steal a large-screen TV from a neighbor's house. The friend wasangry with the young man and decided to use the opportunity to get even with him by having him arrested. Thefriend said he would help, and that night, he drove the young man to the neighbor's house. The young manbroke in while the friend remained outside. The friend called the police on his cell phone and then drove away. Police officers arrived at the scene just as the young man was carrying the TV out the back door. The friend is guilty of what offense in a common law jurisdiction?\",\n \"options\": {\n \"A\": \"Conspiracy.\",\n \"B\": \"Conspiracy and larceny.\",\n \"C\": \"Criminal facilitation.\",\n \"D\": \"No crime.\",\n \"E\": \"Conspiracy and accessory to burglary.\",\n \"F\": \"Solicitation to commit a crime.\",\n \"G\": \"Burglary.\",\n \"H\": \"Aiding and abetting.\",\n \"I\": \"Accessory to burglary.\",\n \"J\": \"Larceny.\"\n }\n}
input
question.json
{
"question": "A young man suggested to his friend that they steal a large-screen TV from a neighbor's house. The friend wasangry with the young man and decided to use the opportunity to get even with him by having him arrested. Thefriend said he would help, and that night, he drove the young man to the neighbor's house. The young manbroke in while the friend remained outside. The friend called the police on his cell phone and then drove away. Police officers arrived at the scene just as the young man was carrying the TV out the back door. The friend is guilty of what offense in a common law jurisdiction?",
"options": {
"A": "Conspiracy.",
"B": "Conspiracy and larceny.",
"C": "Criminal facilitation.",
"D": "No crime.",
"E": "Conspiracy and accessory to burglary.",
"F": "Solicitation to commit a crime.",
"G": "Burglary.",
"H": "Aiding and abetting.",
"I": "Accessory to burglary.",
"J": "Larceny."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_033/ files.
Scored by a judge this pinned commit does not carry.
▸case_034Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Steam at 15.3 KN/m^2 pressure condenses at a rate of 25 kg/hr on a vertical pipe, 56 mm OD. The temperature drop across the film is 5°C. Calculate a) the coefficient of heat transfer, and b) the required length of the pipe\",\n \"options\": {\n \"A\": \"Coefficient of heat transfer = 6000 W/m^2-°C, Length of the pipe = 4.2 m\",\n \"B\": \"Coefficient of heat transfer = 5000 W/m^2-°C, Length of the pipe = 3.0 m\",\n \"C\": \"Coefficient of heat transfer = 4595 W/m^2-°C, Length of the pipe = 2.5 m\",\n \"D\": \"Coefficient of heat transfer = 5514 W/m^2-°C, Length of the pipe = 3.4 m\"\n }\n}
input
question.json
{
"question": "Steam at 15.3 KN/m^2 pressure condenses at a rate of 25 kg/hr on a vertical pipe, 56 mm OD. The temperature drop across the film is 5°C. Calculate a) the coefficient of heat transfer, and b) the required length of the pipe",
"options": {
"A": "Coefficient of heat transfer = 6000 W/m^2-°C, Length of the pipe = 4.2 m",
"B": "Coefficient of heat transfer = 5000 W/m^2-°C, Length of the pipe = 3.0 m",
"C": "Coefficient of heat transfer = 4595 W/m^2-°C, Length of the pipe = 2.5 m",
"D": "Coefficient of heat transfer = 5514 W/m^2-°C, Length of the pipe = 3.4 m"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_034/ files.
Scored by a judge this pinned commit does not carry.
▸case_035Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A lumber mill contracted to deliver one thousand8-foot sheets of plywood to a home improvement store on the first of each month for 10 consecutive months starting June 1. The June, July, and August shipments were delivered on time and duly paid for. Then, on August 16, the lumber mill informed the store that the mill would not be able to meet the September 1 delivery date, because its lumber reserve had been destroyed by a forest fire. The mill then asked the store to excuse the mill from further performance. The store refused and demanded that the remaining shipments be delivered on time. When the September shipment failed to arrive, the store immediately brought suit for breach of contract. How would the court hold?\",\n \"options\": {\n \"A\": \"Judgment for the store, because the mill failed to have a contingency plan in place.\",\n \"B\": \"Judgment for the mill, because their performance would be discharged by impossibility.\",\n \"C\": \"Judgment for the store, because the mill should have foreseen such a contingency occurring.\",\n \"D\": \"Judgment for the store, because the mill's duties of performance would not be excused.\",\n \"E\": \"Judgment for the mill, because the forest fire caused an unforeseeable hindrance in performance.\",\n \"F\": \"Judgment for the mill, because their performance would be discharged by frustration of purpose.\",\n \"G\": \"Judgment for the mill, because the forest fire was an act of God and beyond their control.\",\n \"H\": \"Judgment for the store, because the mill had a contractual obligation to deliver the goods.\",\n \"I\": \"Judgment for the mill, because they had informed the store about the issue in advance.\",\n \"J\": \"Judgment for the store, because the mill could have sourced lumber from another location.\"\n }\n}
input
question.json
{
"question": "A lumber mill contracted to deliver one thousand8-foot sheets of plywood to a home improvement store on the first of each month for 10 consecutive months starting June 1. The June, July, and August shipments were delivered on time and duly paid for. Then, on August 16, the lumber mill informed the store that the mill would not be able to meet the September 1 delivery date, because its lumber reserve had been destroyed by a forest fire. The mill then asked the store to excuse the mill from further performance. The store refused and demanded that the remaining shipments be delivered on time. When the September shipment failed to arrive, the store immediately brought suit for breach of contract. How would the court hold?",
"options": {
"A": "Judgment for the store, because the mill failed to have a contingency plan in place.",
"B": "Judgment for the mill, because their performance would be discharged by impossibility.",
"C": "Judgment for the store, because the mill should have foreseen such a contingency occurring.",
"D": "Judgment for the store, because the mill's duties of performance would not be excused.",
"E": "Judgment for the mill, because the forest fire caused an unforeseeable hindrance in performance.",
"F": "Judgment for the mill, because their performance would be discharged by frustration of purpose.",
"G": "Judgment for the mill, because the forest fire was an act of God and beyond their control.",
"H": "Judgment for the store, because the mill had a contractual obligation to deliver the goods.",
"I": "Judgment for the mill, because they had informed the store about the issue in advance.",
"J": "Judgment for the store, because the mill could have sourced lumber from another location."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_035/ files.
Scored by a judge this pinned commit does not carry.
▸case_036Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Find the cutoff frequency of the first higher-order mode for anair-dielectric 50-ohm coaxial line whose inner conductor hasa radius of 2 mm.\",\n \"options\": {\n \"A\": \"10 GHz\",\n \"B\": \"14.5 GHz\",\n \"C\": \"22 GHz\",\n \"D\": \"11 GHz\",\n \"E\": \"25 GHz\",\n \"F\": \"18 GHz\",\n \"G\": \"20 GHz\",\n \"H\": \"12 GHz\",\n \"I\": \"16 GHz\",\n \"J\": \"8 GHz\"\n }\n}
input
question.json
{
"question": "Find the cutoff frequency of the first higher-order mode for anair-dielectric 50-ohm coaxial line whose inner conductor hasa radius of 2 mm.",
"options": {
"A": "10 GHz",
"B": "14.5 GHz",
"C": "22 GHz",
"D": "11 GHz",
"E": "25 GHz",
"F": "18 GHz",
"G": "20 GHz",
"H": "12 GHz",
"I": "16 GHz",
"J": "8 GHz"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_036/ files.
Scored by a judge this pinned commit does not carry.
▸case_037Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A coil that has a resistance of 0.05 ohms is connected to a cellthat develops anemfof 1.5 volts. Find the current flowingif the internal resistance of the cell is (a) 0.1ohm,and (b) 0.01 ohm.\",\n \"options\": {\n \"A\": \"(a) 7.5 amp, (b) 50 amp\",\n \"B\": \"(a) 14 amp, (b) 28 amp\",\n \"C\": \"(a) 12 amp, (b) 24 amp\",\n \"D\": \"(a) 20 amp, (b) 5 amp\",\n \"E\": \"(a) 5 amp, (b) 20 amp\",\n \"F\": \"(a) 30 amp, (b) 15 amp\",\n \"G\": \"(a) 15 amp, (b) 30 amp\",\n \"H\": \"(a) 25 amp, (b) 10 amp\",\n \"I\": \"(a) 8 amp, (b) 40 amp\",\n \"J\": \"(a) 10 amp, (b) 25 amp\"\n }\n}
input
question.json
{
"question": "A coil that has a resistance of 0.05 ohms is connected to a cellthat develops anemfof 1.5 volts. Find the current flowingif the internal resistance of the cell is (a) 0.1ohm,and (b) 0.01 ohm.",
"options": {
"A": "(a) 7.5 amp, (b) 50 amp",
"B": "(a) 14 amp, (b) 28 amp",
"C": "(a) 12 amp, (b) 24 amp",
"D": "(a) 20 amp, (b) 5 amp",
"E": "(a) 5 amp, (b) 20 amp",
"F": "(a) 30 amp, (b) 15 amp",
"G": "(a) 15 amp, (b) 30 amp",
"H": "(a) 25 amp, (b) 10 amp",
"I": "(a) 8 amp, (b) 40 amp",
"J": "(a) 10 amp, (b) 25 amp"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_037/ files.
Scored by a judge this pinned commit does not carry.
▸case_038Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Ethane reacts with astoichiometricamount of air by the followingcombustion equation : C_2H_6 + 3.5O_2 + 3.5(3.76)N_2\\\\rightharpoonup 2CO_2 + 3H_2O + 3.5(3.76)N_2. Using the following enthalpy data, calculate the amount of heattransfer per mole of fuel if ethane enters at 60 C, the air entersat 4.4°C and the products leave at 449°C.\",\n \"options\": {\n \"A\": \"115,500 kcal/mole\",\n \"B\": \"130,000 kcal/mole\",\n \"C\": \"110,950 kcal/mole\",\n \"D\": \"135,800 kcal/mole\",\n \"E\": \"120,000 kcal/mole\",\n \"F\": \"150,000 kcal/mole\",\n \"G\": \"138,250 kcal/mole\",\n \"H\": \"145,000 kcal/mole\",\n \"I\": \"128,700 kcal/mole\",\n \"J\": \"126,950 kcal/mole\"\n }\n}
input
question.json
{
"question": "Ethane reacts with astoichiometricamount of air by the followingcombustion equation : C_2H_6 + 3.5O_2 + 3.5(3.76)N_2\\rightharpoonup 2CO_2 + 3H_2O + 3.5(3.76)N_2. Using the following enthalpy data, calculate the amount of heattransfer per mole of fuel if ethane enters at 60 C, the air entersat 4.4°C and the products leave at 449°C.",
"options": {
"A": "115,500 kcal/mole",
"B": "130,000 kcal/mole",
"C": "110,950 kcal/mole",
"D": "135,800 kcal/mole",
"E": "120,000 kcal/mole",
"F": "150,000 kcal/mole",
"G": "138,250 kcal/mole",
"H": "145,000 kcal/mole",
"I": "128,700 kcal/mole",
"J": "126,950 kcal/mole"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_038/ files.
Scored by a judge this pinned commit does not carry.
▸case_039Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A bank loaned a company $1,500,000 for the manufacture of a widget control system. As a condition of the loan, a majority shareholder in the company agreed in writing to personally guarantee the loan. Thereafter, the company defaulted on the loan and entered into a repayment agreement with the bank. This agreement provided that the bank would \\\"use maximum efforts in selling the company's assets at the highest possible price. \\\" The bank proceeded to sell the company's assets, discharging the indebtedness of the widget company. Later, it was ascertained that the bank did not realize the \\\"highest possible price\\\" in administering the sale of the company's assets. Consequently, the shareholder was forced into bankruptcy and lost her entire investment in the company. The shareholder brings an appropriate action against the bank to recover her investment in the company. She will most likely\",\n \"options\": {\n \"A\": \"prevail, because the bank's inability to get the highest possible price directly resulted in the shareholder's bankruptcy.\",\n \"B\": \"prevail, because the bank did not act in the best interest of the shareholder.\",\n \"C\": \"not prevail, because the shareholder agreed to personally guarantee the loan.\",\n \"D\": \"prevail, because the bank breached its fiduciary duty to the shareholder under the terms of the contract between the bank and the company.\",\n \"E\": \"not prevail, because the bank's sale of the company's assets discharged whatever contractual relationship existed between the shareholder and the bank.\",\n \"F\": \"prevail, because the bank did not use maximum efforts in selling the company's assets.\",\n \"G\": \"prevail, because the bank did not realize the highest possible price from the sale of the cornpany's assets.\",\n \"H\": \"not prevail, because the shareholder was an incidental beneficiary of the bank's sale of the company's assets.\",\n \"I\": \"not prevail, because the bank has the right to sell the company's assets as it sees fit.\",\n \"J\": \"not prevail, because the bank was only required to use maximum efforts, not necessarily achieve the highest possible price.\"\n }\n}
input
question.json
{
"question": "A bank loaned a company $1,500,000 for the manufacture of a widget control system. As a condition of the loan, a majority shareholder in the company agreed in writing to personally guarantee the loan. Thereafter, the company defaulted on the loan and entered into a repayment agreement with the bank. This agreement provided that the bank would \"use maximum efforts in selling the company's assets at the highest possible price. \" The bank proceeded to sell the company's assets, discharging the indebtedness of the widget company. Later, it was ascertained that the bank did not realize the \"highest possible price\" in administering the sale of the company's assets. Consequently, the shareholder was forced into bankruptcy and lost her entire investment in the company. The shareholder brings an appropriate action against the bank to recover her investment in the company. She will most likely",
"options": {
"A": "prevail, because the bank's inability to get the highest possible price directly resulted in the shareholder's bankruptcy.",
"B": "prevail, because the bank did not act in the best interest of the shareholder.",
"C": "not prevail, because the shareholder agreed to personally guarantee the loan.",
"D": "prevail, because the bank breached its fiduciary duty to the shareholder under the terms of the contract between the bank and the company.",
"E": "not prevail, because the bank's sale of the company's assets discharged whatever contractual relationship existed between the shareholder and the bank.",
"F": "prevail, because the bank did not use maximum efforts in selling the company's assets.",
"G": "prevail, because the bank did not realize the highest possible price from the sale of the cornpany's assets.",
"H": "not prevail, because the shareholder was an incidental beneficiary of the bank's sale of the company's assets.",
"I": "not prevail, because the bank has the right to sell the company's assets as it sees fit.",
"J": "not prevail, because the bank was only required to use maximum efforts, not necessarily achieve the highest possible price."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_039/ files.
Scored by a judge this pinned commit does not carry.
▸case_040Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"An accounting professor has been called as a defense expert witness in a white-collar criminal case. He testified that accounting records conclusively demonstrated that the defendant had not committed any type of fraud. On cross-examination of the professor, the prosecuting attorney asked him if he was familiar with a particular treatise. The professor responded affirmatively. The attorney then asked him if the treatise was accepted as authoritative and if he was aware of the conclusions regarding the ability of accountants to detect fraud. The professor responded in the affirmative. The prosecutor now attempts to read the following section of the treatise into evidence: \\\"It is virtually impossible to state with conclusive certainty the existence, or lack thereof, of fraud in the majority of white-collar crime cases, given the sophistication of defendants and the variety of accounting methods available. \\\"On objection by the defendant's attorney, the trial judge will\",\n \"options\": {\n \"A\": \"sustain the objection, because the professor did not rely on the treatise in making his conclusions.\",\n \"B\": \"sustain the objection, because the statements in the treatise are hearsay not within any recognized exception.\",\n \"C\": \"overrule the objection and permit the prosecutor to ask the professor about the treatise.\",\n \"D\": \"sustain the objection, because the prosecutor's question is leading.\",\n \"E\": \"sustain the objection, because the treatise is not relevant to the professor's testimony.\",\n \"F\": \"overrule the objection, because the treatise is considered a reliable source in the field of accounting.\",\n \"G\": \"overrule the objection, because the professor confirmed the treatise's authority.\",\n \"H\": \"overrule the objection and permit the entire treatise to be introduced into evidence.\",\n \"I\": \"sustain the objection, because the treatise's statements are opinion-based.\",\n \"J\": \"overrule the objection and permit only the contradictory statements to be read into evidence.\"\n }\n}
input
question.json
{
"question": "An accounting professor has been called as a defense expert witness in a white-collar criminal case. He testified that accounting records conclusively demonstrated that the defendant had not committed any type of fraud. On cross-examination of the professor, the prosecuting attorney asked him if he was familiar with a particular treatise. The professor responded affirmatively. The attorney then asked him if the treatise was accepted as authoritative and if he was aware of the conclusions regarding the ability of accountants to detect fraud. The professor responded in the affirmative. The prosecutor now attempts to read the following section of the treatise into evidence: \"It is virtually impossible to state with conclusive certainty the existence, or lack thereof, of fraud in the majority of white-collar crime cases, given the sophistication of defendants and the variety of accounting methods available. \"On objection by the defendant's attorney, the trial judge will",
"options": {
"A": "sustain the objection, because the professor did not rely on the treatise in making his conclusions.",
"B": "sustain the objection, because the statements in the treatise are hearsay not within any recognized exception.",
"C": "overrule the objection and permit the prosecutor to ask the professor about the treatise.",
"D": "sustain the objection, because the prosecutor's question is leading.",
"E": "sustain the objection, because the treatise is not relevant to the professor's testimony.",
"F": "overrule the objection, because the treatise is considered a reliable source in the field of accounting.",
"G": "overrule the objection, because the professor confirmed the treatise's authority.",
"H": "overrule the objection and permit the entire treatise to be introduced into evidence.",
"I": "sustain the objection, because the treatise's statements are opinion-based.",
"J": "overrule the objection and permit only the contradictory statements to be read into evidence."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_040/ files.
Scored by a judge this pinned commit does not carry.
▸case_041Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"On a very cold day in Alaska the temperature is measured to be - 60°C. Assuming atmospheric pressure to be 100kPa, calculate the value of the saturation pressure of the water vapor in theall.\",\n \"options\": {\n \"A\": \"5kPa\",\n \"B\": \"0.0129kPa\",\n \"C\": \"0.2kPa\",\n \"D\": \"0.0005kPa\",\n \"E\": \"100kPa\",\n \"F\": \"0.00109kPa\",\n \"G\": \"11.87kPa\",\n \"H\": \"25kPa\",\n \"I\": \"0.5kPa\",\n \"J\": \"50kPa\"\n }\n}
input
question.json
{
"question": "On a very cold day in Alaska the temperature is measured to be - 60°C. Assuming atmospheric pressure to be 100kPa, calculate the value of the saturation pressure of the water vapor in theall.",
"options": {
"A": "5kPa",
"B": "0.0129kPa",
"C": "0.2kPa",
"D": "0.0005kPa",
"E": "100kPa",
"F": "0.00109kPa",
"G": "11.87kPa",
"H": "25kPa",
"I": "0.5kPa",
"J": "50kPa"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_041/ files.
Scored by a judge this pinned commit does not carry.
▸case_042Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A state has recently enacted a statute making it a misdemeanor for any person to smoke a cigarette, cigar, or pipe in any restaurant, bar, cafe, or other establishment within the state. A defendant was tried in state court for violating the anti-smoking statute. The prosecution's evidence consisted of testimony regarding the defendant's cigarette smoking at a restaurant. During the prosecution's case-in-chief, they called a witness who testified to a prior conviction of the defendant that had been excluded by the judge in a pretrial hearing. The judge immediately granted a mistrial and excused the jury. The state prosecutor appealed the ruling under a statute that permitted prosecutorial appeals in such circumstances. The state won the appeal, but the prosecutor decided not to re-prosecute the defendant. However, the city attorney from the city in which the restaurant was located then sought to prosecute the defendant for the same incident under the city's anti-smoking ordinance, which was identical to the state statute. The city attorney planned to call as her only witness a man who testified at the defendant's first trial. The defendant moved to dismiss on the grounds that the prosecution of the city charge would violate his rights against double jeopardy. The court should\",\n \"options\": {\n \"A\": \"deny the motion, because the judge granted a mistrial.\",\n \"B\": \"grant the motion, because it is the same incident being prosecuted.\",\n \"C\": \"grant the motion, because the city attorney is using the same witness.\",\n \"D\": \"deny the motion, because the first trial resulted in a mistrial.\",\n \"E\": \"grant the motion, because the law and the evidence will be identical at the second trial.\",\n \"F\": \"deny the motion, because the city's anti-smoking ordinance is separate from the state statute.\",\n \"G\": \"grant the motion, because jeopardy attached in the first prosecution.\",\n \"H\": \"deny the motion, because the city has its own jurisdiction.\",\n \"I\": \"grant the motion, because the defendant was not re-prosecuted by the state.\",\n \"J\": \"deny the motion, because the city and the state are separate sovereigns.\"\n }\n}
input
question.json
{
"question": "A state has recently enacted a statute making it a misdemeanor for any person to smoke a cigarette, cigar, or pipe in any restaurant, bar, cafe, or other establishment within the state. A defendant was tried in state court for violating the anti-smoking statute. The prosecution's evidence consisted of testimony regarding the defendant's cigarette smoking at a restaurant. During the prosecution's case-in-chief, they called a witness who testified to a prior conviction of the defendant that had been excluded by the judge in a pretrial hearing. The judge immediately granted a mistrial and excused the jury. The state prosecutor appealed the ruling under a statute that permitted prosecutorial appeals in such circumstances. The state won the appeal, but the prosecutor decided not to re-prosecute the defendant. However, the city attorney from the city in which the restaurant was located then sought to prosecute the defendant for the same incident under the city's anti-smoking ordinance, which was identical to the state statute. The city attorney planned to call as her only witness a man who testified at the defendant's first trial. The defendant moved to dismiss on the grounds that the prosecution of the city charge would violate his rights against double jeopardy. The court should",
"options": {
"A": "deny the motion, because the judge granted a mistrial.",
"B": "grant the motion, because it is the same incident being prosecuted.",
"C": "grant the motion, because the city attorney is using the same witness.",
"D": "deny the motion, because the first trial resulted in a mistrial.",
"E": "grant the motion, because the law and the evidence will be identical at the second trial.",
"F": "deny the motion, because the city's anti-smoking ordinance is separate from the state statute.",
"G": "grant the motion, because jeopardy attached in the first prosecution.",
"H": "deny the motion, because the city has its own jurisdiction.",
"I": "grant the motion, because the defendant was not re-prosecuted by the state.",
"J": "deny the motion, because the city and the state are separate sovereigns."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_042/ files.
Scored by a judge this pinned commit does not carry.
▸case_043Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A shaft distributes mechanical power of 160 hp at a speed of 400 rpm. It is subjected to sudden loading and heavy shock conditions with a maximum bending moment of 32,000 lbs-in. Assuming that the shaft has a keyway and considering the ASME code, calculate the suitable diameter for the shaft.\",\n \"options\": {\n \"A\": \"5 in.\",\n \"B\": \"4 in.\",\n \"C\": \"3 7/8 in.\",\n \"D\": \"3 1/2 in.\",\n \"E\": \"4 1/2 in.\",\n \"F\": \"4 3/4 in.\",\n \"G\": \"4 3/8 in.\",\n \"H\": \"3 3/4 in.\",\n \"I\": \"4 1/4 in.\",\n \"J\": \"4 5/8 in.\"\n }\n}
input
question.json
{
"question": "A shaft distributes mechanical power of 160 hp at a speed of 400 rpm. It is subjected to sudden loading and heavy shock conditions with a maximum bending moment of 32,000 lbs-in. Assuming that the shaft has a keyway and considering the ASME code, calculate the suitable diameter for the shaft.",
"options": {
"A": "5 in.",
"B": "4 in.",
"C": "3 7/8 in.",
"D": "3 1/2 in.",
"E": "4 1/2 in.",
"F": "4 3/4 in.",
"G": "4 3/8 in.",
"H": "3 3/4 in.",
"I": "4 1/4 in.",
"J": "4 5/8 in."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_043/ files.
Scored by a judge this pinned commit does not carry.
▸case_044Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A 35-year-old man with some experience as a truck driver owned a lumber truck. One day, the man set out driving his truck, heavily loaded with lumber, down a mountain road. Sitting next to the man in the passenger seat was a 19-year-old helper. During the course of the trip, when the truck was going down a long hill, the brakes failed. The man shouted to the helper to jump, but the teenager refused to do so and shouted back to the man that he should try to steer the truck down the hill. The man then opened the door on the passenger's side of the truck and negligently pushed the helper out. The helper, who suffered a broken leg, was rushed to the hospital where he was treated for his injury. As the helper was recuperating, a nurse inadvertently mixed up his chart with that of the female patient in the next room. The nurse gave the helper a fertility pill that made him sterile. If the helper brings suit against the man to recover damages for his sterility, the man will\",\n \"options\": {\n \"A\": \"not recover, because the man had no control over the hospital's actions.\",\n \"B\": \"recover, because the man failed to properly instruct the helper on emergency procedures.\",\n \"C\": \"not recover, because the man's actions were intended to save the helper's life.\",\n \"D\": \"not recover, because the nurse's negligence in giving the wrong medication is unrelated to the man's actions.\",\n \"E\": \"recover, because the man was responsible for the helper's safety while in the truck.\",\n \"F\": \"recover, because the man was negligent in his maintenance of the truck's brakes.\",\n \"G\": \"recover, because the man was negligent in pushing the helper out of the truck.\",\n \"H\": \"not recover, because it is not foreseeable that a patient would be injured in such a manner.\",\n \"I\": \"recover, because it is foreseeable that a hospital can be negligent in its care of patients.\"\n }\n}
input
question.json
{
"question": "A 35-year-old man with some experience as a truck driver owned a lumber truck. One day, the man set out driving his truck, heavily loaded with lumber, down a mountain road. Sitting next to the man in the passenger seat was a 19-year-old helper. During the course of the trip, when the truck was going down a long hill, the brakes failed. The man shouted to the helper to jump, but the teenager refused to do so and shouted back to the man that he should try to steer the truck down the hill. The man then opened the door on the passenger's side of the truck and negligently pushed the helper out. The helper, who suffered a broken leg, was rushed to the hospital where he was treated for his injury. As the helper was recuperating, a nurse inadvertently mixed up his chart with that of the female patient in the next room. The nurse gave the helper a fertility pill that made him sterile. If the helper brings suit against the man to recover damages for his sterility, the man will",
"options": {
"A": "not recover, because the man had no control over the hospital's actions.",
"B": "recover, because the man failed to properly instruct the helper on emergency procedures.",
"C": "not recover, because the man's actions were intended to save the helper's life.",
"D": "not recover, because the nurse's negligence in giving the wrong medication is unrelated to the man's actions.",
"E": "recover, because the man was responsible for the helper's safety while in the truck.",
"F": "recover, because the man was negligent in his maintenance of the truck's brakes.",
"G": "recover, because the man was negligent in pushing the helper out of the truck.",
"H": "not recover, because it is not foreseeable that a patient would be injured in such a manner.",
"I": "recover, because it is foreseeable that a hospital can be negligent in its care of patients."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_044/ files.
Scored by a judge this pinned commit does not carry.
▸case_045Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A 50 V pulse is incident on a 30 ohm load in a line with characteristic resistance of 50 ohms. Find the following a) Incident Current Pulse, b) Reflection Coefficient, c) Reflected Voltage Pulse, d) Reflected Current Pulse, e) Net voltage and current at the load.\",\n \"options\": {\n \"A\": \"Incident Current Pulse: 1 Ampere, Reflection Coefficient: 0, Reflected Voltage Pulse: 0V, Reflected Current Pulse: 0 Amperes, Net voltage at the load: 50V, Net current at the load: 1 Ampere\",\n \"B\": \"Incident Current Pulse: 1.1 Ampere, Reflection Coefficient: -0.1, Reflected Voltage Pulse: 5V, Reflected Current Pulse: -0.11 Amperes, Net voltage at the load: 45V, Net current at the load: 1.21 Amperes\",\n \"C\": \"Incident Current Pulse: 1.2 Ampere, Reflection Coefficient: 0.2, Reflected Voltage Pulse: 10V, Reflected Current Pulse: 0.24 Amperes, Net voltage at the load: 40V, Net current at the load: 1.44 Amperes\",\n \"D\": \"Incident Current Pulse: 2 Amperes, Reflection Coefficient: -0.5, Reflected Voltage Pulse: 15V, Reflected Current Pulse: -0.3 Amperes, Net voltage at the load: 35V, Net current at the load: 1.5 Amperes\",\n \"E\": \"Incident Current Pulse: 0.8 Ampere, Reflection Coefficient: -0.4, Reflected Voltage Pulse: 20V, Reflected Current Pulse: -0.32 Amperes, Net voltage at the load: 30V, Net current at the load: 1.12 Amperes\",\n \"F\": \"Incident Current Pulse: 0.5 Ampere, Reflection Coefficient: -0.1, Reflected Voltage Pulse: 10V, Reflected Current Pulse: -0.2 Amperes, Net voltage at the load: 40V, Net current at the load: 1 Amperes\",\n \"G\": \"Incident Current Pulse: 0.6 Ampere, Reflection Coefficient: -0.2, Reflected Voltage Pulse: 5V, Reflected Current Pulse: -0.1 Amperes, Net voltage at the load: 45V, Net current at the load: 0.7 Amperes\",\n \"H\": \"Incident Current Pulse: 1 Ampere, Reflection Coefficient: -0.25, Reflected Voltage Pulse: 12.5V, Reflected Current Pulse: -0.25 Amperes, Net voltage at the load: 37.5V, Net current at the load: 1.25 Amperes\",\n \"I\": \"Incident Current Pulse: 0.75 Ampere, Reflection Coefficient: 0.4, Reflected Voltage Pulse: 20V, Reflected Current Pulse: 0.3 Amperes, Net voltage at the load: 30V, Net current at the load: 1.05 Amperes\",\n \"J\": \"Incident Current Pulse: 1.5 Ampere, Reflection Coefficient: -0.3, Reflected Voltage Pulse: 15V, Reflected Current Pulse: -0.35 Amperes, Net voltage at the load: 35V, Net current at the load: 1.35 Amperes\"\n }\n}
input
question.json
{
"question": "A 50 V pulse is incident on a 30 ohm load in a line with characteristic resistance of 50 ohms. Find the following a) Incident Current Pulse, b) Reflection Coefficient, c) Reflected Voltage Pulse, d) Reflected Current Pulse, e) Net voltage and current at the load.",
"options": {
"A": "Incident Current Pulse: 1 Ampere, Reflection Coefficient: 0, Reflected Voltage Pulse: 0V, Reflected Current Pulse: 0 Amperes, Net voltage at the load: 50V, Net current at the load: 1 Ampere",
"B": "Incident Current Pulse: 1.1 Ampere, Reflection Coefficient: -0.1, Reflected Voltage Pulse: 5V, Reflected Current Pulse: -0.11 Amperes, Net voltage at the load: 45V, Net current at the load: 1.21 Amperes",
"C": "Incident Current Pulse: 1.2 Ampere, Reflection Coefficient: 0.2, Reflected Voltage Pulse: 10V, Reflected Current Pulse: 0.24 Amperes, Net voltage at the load: 40V, Net current at the load: 1.44 Amperes",
"D": "Incident Current Pulse: 2 Amperes, Reflection Coefficient: -0.5, Reflected Voltage Pulse: 15V, Reflected Current Pulse: -0.3 Amperes, Net voltage at the load: 35V, Net current at the load: 1.5 Amperes",
"E": "Incident Current Pulse: 0.8 Ampere, Reflection Coefficient: -0.4, Reflected Voltage Pulse: 20V, Reflected Current Pulse: -0.32 Amperes, Net voltage at the load: 30V, Net current at the load: 1.12 Amperes",
"F": "Incident Current Pulse: 0.5 Ampere, Reflection Coefficient: -0.1, Reflected Voltage Pulse: 10V, Reflected Current Pulse: -0.2 Amperes, Net voltage at the load: 40V, Net current at the load: 1 Amperes",
"G": "Incident Current Pulse: 0.6 Ampere, Reflection Coefficient: -0.2, Reflected Voltage Pulse: 5V, Reflected Current Pulse: -0.1 Amperes, Net voltage at the load: 45V, Net current at the load: 0.7 Amperes",
"H": "Incident Current Pulse: 1 Ampere, Reflection Coefficient: -0.25, Reflected Voltage Pulse: 12.5V, Reflected Current Pulse: -0.25 Amperes, Net voltage at the load: 37.5V, Net current at the load: 1.25 Amperes",
"I": "Incident Current Pulse: 0.75 Ampere, Reflection Coefficient: 0.4, Reflected Voltage Pulse: 20V, Reflected Current Pulse: 0.3 Amperes, Net voltage at the load: 30V, Net current at the load: 1.05 Amperes",
"J": "Incident Current Pulse: 1.5 Ampere, Reflection Coefficient: -0.3, Reflected Voltage Pulse: 15V, Reflected Current Pulse: -0.35 Amperes, Net voltage at the load: 35V, Net current at the load: 1.35 Amperes"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_045/ files.
Scored by a judge this pinned commit does not carry.
▸case_046Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Four gallons of gasoline (A) is poured on a surface area of 20 ft^2.Determine the time necessary for the gasoline to evaporateinto still dry air (B). The diffusivity is D_AB = 6.5 ft^2 / hr. Evaporation occurs through a 6 in. filmat65°F. The gasoline has a vapor pressure of 2psia.\",\n \"options\": {\n \"A\": \"6.5 hr\",\n \"B\": \"7.2 hr\",\n \"C\": \"2.5 hr\",\n \"D\": \"14.1 hr\",\n \"E\": \"9.5 hr\",\n \"F\": \"12.8 hr\",\n \"G\": \"4.7 hr\",\n \"H\": \"3.3 hr\",\n \"I\": \"8.39 hr\",\n \"J\": \"10.2 hr\"\n }\n}
input
question.json
{
"question": "Four gallons of gasoline (A) is poured on a surface area of 20 ft^2.Determine the time necessary for the gasoline to evaporateinto still dry air (B). The diffusivity is D_AB = 6.5 ft^2 / hr. Evaporation occurs through a 6 in. filmat65°F. The gasoline has a vapor pressure of 2psia.",
"options": {
"A": "6.5 hr",
"B": "7.2 hr",
"C": "2.5 hr",
"D": "14.1 hr",
"E": "9.5 hr",
"F": "12.8 hr",
"G": "4.7 hr",
"H": "3.3 hr",
"I": "8.39 hr",
"J": "10.2 hr"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_046/ files.
Scored by a judge this pinned commit does not carry.
▸case_047Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Saturated steam, at 100°F, condenses on the outer surface ofa vertical pipe, 1 ft long and 6 in OD, maintained at 80°F. Calculate a)theaverage coefficient of heat transfer, b)therate of heat transfer, and c)themass flow rate of the condensate, assuming laminar flow.\",\n \"options\": {\n \"A\": \"a) 1,250 Btu/hr-ft^2-°F, b) 41,000 Btu/hr, c) 42lbm/hr\",\n \"B\": \"a) 1,500 Btu/hr-ft^2-°F, b) 45,000 Btu/hr, c) 45lbm/hr\",\n \"C\": \"a) 1,357.2 Btu/hr-ft^2-°F, b) 42,638 Btu/hr, c) 41lbm/hr\",\n \"D\": \"a) 1,200 Btu/hr-ft^2-°F, b) 40,000 Btu/hr, c) 40lbm/hr\"\n }\n}
input
question.json
{
"question": "Saturated steam, at 100°F, condenses on the outer surface ofa vertical pipe, 1 ft long and 6 in OD, maintained at 80°F. Calculate a)theaverage coefficient of heat transfer, b)therate of heat transfer, and c)themass flow rate of the condensate, assuming laminar flow.",
"options": {
"A": "a) 1,250 Btu/hr-ft^2-°F, b) 41,000 Btu/hr, c) 42lbm/hr",
"B": "a) 1,500 Btu/hr-ft^2-°F, b) 45,000 Btu/hr, c) 45lbm/hr",
"C": "a) 1,357.2 Btu/hr-ft^2-°F, b) 42,638 Btu/hr, c) 41lbm/hr",
"D": "a) 1,200 Btu/hr-ft^2-°F, b) 40,000 Btu/hr, c) 40lbm/hr"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_047/ files.
Scored by a judge this pinned commit does not carry.
▸case_048Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Loretta Landlord owned a commercial building in a commercial strip surrounded by a residential community. One of her largest spaces was rented on a month-to-month basis at $500 per month by a dance troop which held performances in the facility. The director of the dance troop moved to New York for a starring role Off-Broadway. Unfortunately the troop was unable to find another dance director, so it folded and vacated the space. Sally Sleazy learned of the closure and decided the location would be perfect for a topless strip joint. Sally rented the space from Loretta under a two-year lease at $1,000 per month without disclosing her intended use of the space. One month after the strip joint opened, the County Commissioners passed a new law prohibiting topless strip joints in that neighborhood. Sally told Loretta the new law made it impossible to operate the business, so she was leaving. If Loretta then sued Sally for breach of the lease contract, the likely outcome is a judgment for\",\n \"options\": {\n \"A\": \"Loretta, only if she can prove that Sally knew about the upcoming law change.\",\n \"B\": \"Loretta, if she can prove that Sally intended to breach the contract from the start.\",\n \"C\": \"Sally, because the change in law was beyond her control.\",\n \"D\": \"Sally, because the law change is a valid reason for terminating the lease.\",\n \"E\": \"Sally, because the law made performance of the lease contract impossible.\",\n \"F\": \"Sally, because the lease contract was discharged under impossibility of performance.\",\n \"G\": \"Loretta, only if she did not know of the use Sally intended for the space.\",\n \"H\": \"Loretta, because the new law was passed after the lease was executed.\",\n \"I\": \"Sally, because the law change made the purpose of the lease void.\"\n }\n}
input
question.json
{
"question": "Loretta Landlord owned a commercial building in a commercial strip surrounded by a residential community. One of her largest spaces was rented on a month-to-month basis at $500 per month by a dance troop which held performances in the facility. The director of the dance troop moved to New York for a starring role Off-Broadway. Unfortunately the troop was unable to find another dance director, so it folded and vacated the space. Sally Sleazy learned of the closure and decided the location would be perfect for a topless strip joint. Sally rented the space from Loretta under a two-year lease at $1,000 per month without disclosing her intended use of the space. One month after the strip joint opened, the County Commissioners passed a new law prohibiting topless strip joints in that neighborhood. Sally told Loretta the new law made it impossible to operate the business, so she was leaving. If Loretta then sued Sally for breach of the lease contract, the likely outcome is a judgment for",
"options": {
"A": "Loretta, only if she can prove that Sally knew about the upcoming law change.",
"B": "Loretta, if she can prove that Sally intended to breach the contract from the start.",
"C": "Sally, because the change in law was beyond her control.",
"D": "Sally, because the law change is a valid reason for terminating the lease.",
"E": "Sally, because the law made performance of the lease contract impossible.",
"F": "Sally, because the lease contract was discharged under impossibility of performance.",
"G": "Loretta, only if she did not know of the use Sally intended for the space.",
"H": "Loretta, because the new law was passed after the lease was executed.",
"I": "Sally, because the law change made the purpose of the lease void."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_048/ files.
Scored by a judge this pinned commit does not carry.
▸case_049Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"On Thursday, March 7, an owner's home was destroyed by fire. The owner's house was insured against such fire loss by an insurance company. According to the policy contract, the insured was required to notify the insurance company within 10 days following any loss due to fire or other catastrophe. The owner sued the insurance company to recover $750,000, which was the face amount that his home was insured for. The insurance company admitted that the owner's house was insured for that amount, though the company denied liability because it claimed that it had not received notice of the fire within the 10-day period as required by the policy. At trial, the owner calls his bookkeeper to testify. The bookkeeper proposes to testify that on March 11, the owner handed her an envelope and told her it contained a notice form for the insurance company, and that she mailed the envelope on his behalf. If the bookkeeper intends to establish that the envelope that the owner handed her contained the notice of fire loss, her proposed testimony is\",\n \"options\": {\n \"A\": \"admissible, because the owner's statement to her is non-hearsay.\",\n \"B\": \"inadmissible, as the bookkeeper's testimony is considered second-hand information.\",\n \"C\": \"admissible, because a letter properly mailed is presumed received in due course.\",\n \"D\": \"inadmissible, because it is the responsibility of the owner, not the bookkeeper, to ensure the insurance company received the notice.\",\n \"E\": \"inadmissible, because the owner's statement to the bookkeeper is not the best evidence.\",\n \"F\": \"admissible, as the bookkeeper's testimony provides proof of the owner's attempt to notify the insurance company.\",\n \"G\": \"inadmissible, because the owner's statement to the bookkeeper is hearsay not within any exception.\",\n \"H\": \"inadmissible, because the bookkeeper cannot confirm the contents of the envelope.\",\n \"I\": \"admissible, because the owner's statement is considered a verbal act.\",\n \"J\": \"admissible, because the bookkeeper is a credible witness.\"\n }\n}
input
question.json
{
"question": "On Thursday, March 7, an owner's home was destroyed by fire. The owner's house was insured against such fire loss by an insurance company. According to the policy contract, the insured was required to notify the insurance company within 10 days following any loss due to fire or other catastrophe. The owner sued the insurance company to recover $750,000, which was the face amount that his home was insured for. The insurance company admitted that the owner's house was insured for that amount, though the company denied liability because it claimed that it had not received notice of the fire within the 10-day period as required by the policy. At trial, the owner calls his bookkeeper to testify. The bookkeeper proposes to testify that on March 11, the owner handed her an envelope and told her it contained a notice form for the insurance company, and that she mailed the envelope on his behalf. If the bookkeeper intends to establish that the envelope that the owner handed her contained the notice of fire loss, her proposed testimony is",
"options": {
"A": "admissible, because the owner's statement to her is non-hearsay.",
"B": "inadmissible, as the bookkeeper's testimony is considered second-hand information.",
"C": "admissible, because a letter properly mailed is presumed received in due course.",
"D": "inadmissible, because it is the responsibility of the owner, not the bookkeeper, to ensure the insurance company received the notice.",
"E": "inadmissible, because the owner's statement to the bookkeeper is not the best evidence.",
"F": "admissible, as the bookkeeper's testimony provides proof of the owner's attempt to notify the insurance company.",
"G": "inadmissible, because the owner's statement to the bookkeeper is hearsay not within any exception.",
"H": "inadmissible, because the bookkeeper cannot confirm the contents of the envelope.",
"I": "admissible, because the owner's statement is considered a verbal act.",
"J": "admissible, because the bookkeeper is a credible witness."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_049/ files.
Scored by a judge this pinned commit does not carry.
▸case_050Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A model pump has an impeller of 6 in. diameter. When runningat 1200 rpm, the pump delivers 2cfsagainst a head of16 ft. A similar pump is required to discharge 40cfsat 600 rpm.What should be the diameter of this pump, and whathead will it develop?\",\n \"options\": {\n \"A\": \"Diameter: 1.8 ft, Head: 36 ft\",\n \"B\": \"Diameter: 2.2 ft, Head: 30 ft\",\n \"C\": \"Diameter: 1.71 ft, Head: 46.8 ft\",\n \"D\": \"Diameter: 2.5 ft, Head: 50 ft\",\n \"E\": \"Diameter: 2.4 ft, Head: 55 ft\",\n \"F\": \"Diameter: 3 ft, Head: 60 ft\",\n \"G\": \"Diameter: 1.2 ft, Head: 20 ft\",\n \"H\": \"Diameter: 2 ft, Head: 45 ft\",\n \"I\": \"Diameter: 1.5 ft, Head: 40 ft\",\n \"J\": \"Diameter: 1.9 ft, Head: 42 ft\"\n }\n}
input
question.json
{
"question": "A model pump has an impeller of 6 in. diameter. When runningat 1200 rpm, the pump delivers 2cfsagainst a head of16 ft. A similar pump is required to discharge 40cfsat 600 rpm.What should be the diameter of this pump, and whathead will it develop?",
"options": {
"A": "Diameter: 1.8 ft, Head: 36 ft",
"B": "Diameter: 2.2 ft, Head: 30 ft",
"C": "Diameter: 1.71 ft, Head: 46.8 ft",
"D": "Diameter: 2.5 ft, Head: 50 ft",
"E": "Diameter: 2.4 ft, Head: 55 ft",
"F": "Diameter: 3 ft, Head: 60 ft",
"G": "Diameter: 1.2 ft, Head: 20 ft",
"H": "Diameter: 2 ft, Head: 45 ft",
"I": "Diameter: 1.5 ft, Head: 40 ft",
"J": "Diameter: 1.9 ft, Head: 42 ft"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_050/ files.
Scored by a judge this pinned commit does not carry.
▸case_051Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Before a proposal to merge two towns was placed on a ballot, the state attorney general issued an advisory opinion stating that the measure did not, in her opinion, violate any statutory or constitutional provisions. Thereafter, the proposal was placed on the ballot and was overwhelmingly passed by the voters in both towns. After the election, but before the merger had officially been carried out, two taxpayers from one of the towns initiated suit to enjoin the unification, attacking the constitutionality of the proposal. The suit reached the state supreme court and was found to be constitutional under both state and federal law. The plaintiffs now file a motion seeking to have thiscase reviewed by the U. S. Supreme Court. The Court may\",\n \"options\": {\n \"A\": \"not hear the case, because it was decided on independent state grounds.\",\n \"B\": \"not hear the case, because the U.S. Supreme Court does not have jurisdiction over state matters\",\n \"C\": \"not hear the case, but may have it remanded to federal district court.\",\n \"D\": \"hear the case because it involves a constitutional issue\",\n \"E\": \"hear the federal issues involved, but decline to rule on the state issue.\",\n \"F\": \"hear the case, but only rule on the constitutionality of the merger\",\n \"G\": \"hear the case on the grounds that the plaintiffs have a right to a federal hearing\",\n \"H\": \"rely on the advisory opinion rendered by the state attorney general and not hear the case on its merits.\",\n \"I\": \"not hear the case, because the U.S. Supreme Court does not review state supreme court decisions unless federal law is violated.\",\n \"J\": \"not hear the case, because the state supreme court's decision is final\"\n }\n}
input
question.json
{
"question": "Before a proposal to merge two towns was placed on a ballot, the state attorney general issued an advisory opinion stating that the measure did not, in her opinion, violate any statutory or constitutional provisions. Thereafter, the proposal was placed on the ballot and was overwhelmingly passed by the voters in both towns. After the election, but before the merger had officially been carried out, two taxpayers from one of the towns initiated suit to enjoin the unification, attacking the constitutionality of the proposal. The suit reached the state supreme court and was found to be constitutional under both state and federal law. The plaintiffs now file a motion seeking to have thiscase reviewed by the U. S. Supreme Court. The Court may",
"options": {
"A": "not hear the case, because it was decided on independent state grounds.",
"B": "not hear the case, because the U.S. Supreme Court does not have jurisdiction over state matters",
"C": "not hear the case, but may have it remanded to federal district court.",
"D": "hear the case because it involves a constitutional issue",
"E": "hear the federal issues involved, but decline to rule on the state issue.",
"F": "hear the case, but only rule on the constitutionality of the merger",
"G": "hear the case on the grounds that the plaintiffs have a right to a federal hearing",
"H": "rely on the advisory opinion rendered by the state attorney general and not hear the case on its merits.",
"I": "not hear the case, because the U.S. Supreme Court does not review state supreme court decisions unless federal law is violated.",
"J": "not hear the case, because the state supreme court's decision is final"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_051/ files.
Scored by a judge this pinned commit does not carry.
▸case_052Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"If the conductivity and relative permittivity of a medium are 0.3 mho/meter and 60 respectively, assuming that\\\\mu_r= 1, does the medium behave like a conductor or a dielectric at a frequency of: (a) 100 KHz, (b) 3000MHz?\",\n \"options\": {\n \"A\": \"At 100 KHz, its behavior is frequency-independent. At 3000MHz, it behaves like a dielectric.\",\n \"B\": \"At both frequencies, its behavior is frequency-independent, acting neither as a good conductor nor as a dielectric.\",\n \"C\": \"At both frequencies, it behaves like a dielectric.\",\n \"D\": \"At 100 KHz, it behaves like a conductor with high losses. At 3000MHz, its behavior is frequency-independent.\",\n \"E\": \"At 100 KHz, it behaves like a poor conductor. At 3000MHz, it behaves like a dielectric with high losses.\",\n \"F\": \"At 100 KHz, it behaves like a dielectric. At 3000MHz, it behaves like a conductor.\",\n \"G\": \"At 100 KHz, it behaves like a good conductor. At 3000MHz, it behaves like a dielectric.\",\n \"H\": \"At 100 KHz, it behaves like a dielectric with high losses. At 3000MHz, it behaves like a good conductor.\",\n \"I\": \"At 100 KHz, it behaves like a dielectric. At 3000MHz, it behaves like a good conductor.\",\n \"J\": \"At both frequencies, it behaves like a good conductor.\"\n }\n}
input
question.json
{
"question": "If the conductivity and relative permittivity of a medium are 0.3 mho/meter and 60 respectively, assuming that\\mu_r= 1, does the medium behave like a conductor or a dielectric at a frequency of: (a) 100 KHz, (b) 3000MHz?",
"options": {
"A": "At 100 KHz, its behavior is frequency-independent. At 3000MHz, it behaves like a dielectric.",
"B": "At both frequencies, its behavior is frequency-independent, acting neither as a good conductor nor as a dielectric.",
"C": "At both frequencies, it behaves like a dielectric.",
"D": "At 100 KHz, it behaves like a conductor with high losses. At 3000MHz, its behavior is frequency-independent.",
"E": "At 100 KHz, it behaves like a poor conductor. At 3000MHz, it behaves like a dielectric with high losses.",
"F": "At 100 KHz, it behaves like a dielectric. At 3000MHz, it behaves like a conductor.",
"G": "At 100 KHz, it behaves like a good conductor. At 3000MHz, it behaves like a dielectric.",
"H": "At 100 KHz, it behaves like a dielectric with high losses. At 3000MHz, it behaves like a good conductor.",
"I": "At 100 KHz, it behaves like a dielectric. At 3000MHz, it behaves like a good conductor.",
"J": "At both frequencies, it behaves like a good conductor."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_052/ files.
Scored by a judge this pinned commit does not carry.
▸case_053Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Why is it important to separate the concept of punishment from its justification?\",\n \"options\": {\n \"A\": \"Because the concept of punishment has evolved over time.\",\n \"B\": \"Because punishment can be justified in multiple ways.\",\n \"C\": \"Because any definition of punishment should be value-neutral.\",\n \"D\": \"Because the practice of punishment is separate from its justification.\",\n \"E\": \"Because the justification of punishment varies across cultures.\",\n \"F\": \"Because the concept of punishment is often misunderstood.\",\n \"G\": \"Because the concept of punishment is controversial.\",\n \"H\": \"Because the justification of punishment is a matter of personal opinion.\",\n \"I\": \"Because its justification depends on the concept employed.\"\n }\n}
input
question.json
{
"question": "Why is it important to separate the concept of punishment from its justification?",
"options": {
"A": "Because the concept of punishment has evolved over time.",
"B": "Because punishment can be justified in multiple ways.",
"C": "Because any definition of punishment should be value-neutral.",
"D": "Because the practice of punishment is separate from its justification.",
"E": "Because the justification of punishment varies across cultures.",
"F": "Because the concept of punishment is often misunderstood.",
"G": "Because the concept of punishment is controversial.",
"H": "Because the justification of punishment is a matter of personal opinion.",
"I": "Because its justification depends on the concept employed."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_053/ files.
Scored by a judge this pinned commit does not carry.
▸case_054Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A buyer signed a contract to purchase a tract of land from a seller. The contract was signed on May 1, 2006, and the closing is set for June 15, 2006. The land was located in a jurisdiction that has in force the following statute:Statute of Limitations \u0080\u0094\\\"an action to recover the possession of land shall be brought within twenty- one (21) years after the cause thereof accrued, but if a person who is entitled to bring such action at the time the cause accrued is within the age of minority (under 21) or of unsound mind, such action may be brought within ten (10) years after such disability is removed. \\\"This statute was enacted in the jurisdiction in 1930. The land in question had been owned by a rancher in 1960. On September 1, 1960, the rancher died intestate, leaving his nephew as his only heir. The nephew had been born on October 1, 1954. In addition, there is a deed from the rancher's administrator to the seller dated November 1, 1960, which the latter recorded on the same date. During his title search, the buyer learned that the administrator had never obtained any license to sell the land in question; and also he (the administrator) never formally received the approval of any court with respect to the administration of the rancher's estate. Moreover, the buyer ascertained that the seller entered into possession of the land in question on November 1, 1960. On the assumption that there are no additional facts, the buyer should be advised that the seller became or will become the owner of the land in question\",\n \"options\": {\n \"A\": \"on June 15, 2006.\",\n \"B\": \"when the nephew reached the age of 21.\",\n \"C\": \"when the administrator received court approval for the sale.\",\n \"D\": \"on May 1, 2006.\",\n \"E\": \"on September 1, 1960.\",\n \"F\": \"on October 1, 1985.\",\n \"G\": \"when the statute of limitations expired.\",\n \"H\": \"on November 12, 1981.\",\n \"I\": \"when the nephew dies.\",\n \"J\": \"on November 1, 1960.\"\n }\n}
input
question.json
{
"question": "A buyer signed a contract to purchase a tract of land from a seller. The contract was signed on May 1, 2006, and the closing is set for June 15, 2006. The land was located in a jurisdiction that has in force the following statute:Statute of Limitations \"an action to recover the possession of land shall be brought within twenty- one (21) years after the cause thereof accrued, but if a person who is entitled to bring such action at the time the cause accrued is within the age of minority (under 21) or of unsound mind, such action may be brought within ten (10) years after such disability is removed. \"This statute was enacted in the jurisdiction in 1930. The land in question had been owned by a rancher in 1960. On September 1, 1960, the rancher died intestate, leaving his nephew as his only heir. The nephew had been born on October 1, 1954. In addition, there is a deed from the rancher's administrator to the seller dated November 1, 1960, which the latter recorded on the same date. During his title search, the buyer learned that the administrator had never obtained any license to sell the land in question; and also he (the administrator) never formally received the approval of any court with respect to the administration of the rancher's estate. Moreover, the buyer ascertained that the seller entered into possession of the land in question on November 1, 1960. On the assumption that there are no additional facts, the buyer should be advised that the seller became or will become the owner of the land in question",
"options": {
"A": "on June 15, 2006.",
"B": "when the nephew reached the age of 21.",
"C": "when the administrator received court approval for the sale.",
"D": "on May 1, 2006.",
"E": "on September 1, 1960.",
"F": "on October 1, 1985.",
"G": "when the statute of limitations expired.",
"H": "on November 12, 1981.",
"I": "when the nephew dies.",
"J": "on November 1, 1960."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_054/ files.
Scored by a judge this pinned commit does not carry.
▸case_055Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Evaluatedivuat the point (- 1, 1, 2) foru= x^2i+e^xyj+ xyzk(1)\",\n \"options\": {\n \"A\": \"- 3 - e^1\",\n \"B\": \"- 2 - e^-1\",\n \"C\": \"2 + e^-1\",\n \"D\": \"- 3 - e^-1\",\n \"E\": \"-3 + e^1\",\n \"F\": \"-4 + e^-1\",\n \"G\": \"-1 + e^1\",\n \"H\": \"-1 - e^-1\",\n \"I\": \"- 3 + e^-1\",\n \"J\": \"-2 + e^1\"\n }\n}
input
question.json
{
"question": "Evaluatedivuat the point (- 1, 1, 2) foru= x^2i+e^xyj+ xyzk(1)",
"options": {
"A": "- 3 - e^1",
"B": "- 2 - e^-1",
"C": "2 + e^-1",
"D": "- 3 - e^-1",
"E": "-3 + e^1",
"F": "-4 + e^-1",
"G": "-1 + e^1",
"H": "-1 - e^-1",
"I": "- 3 + e^-1",
"J": "-2 + e^1"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_055/ files.
Scored by a judge this pinned commit does not carry.
▸case_056Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Find the inverse Laplace transform of F(s) = [(2s^2 - 4) / {(s - 2)(s + 1)(s - 3)}]\",\n \"options\": {\n \"A\": \"f(t) = [(- 4) / 3]e^-2t - (1 / 6)e^2t + (7 / 2)e^-3t\",\n \"B\": \"f(t) = [(- 4) / 3]e^2t - (1 / 6)e^-t - (7 / 2)e^3t\",\n \"C\": \"f(t) = [(- 4) / 3]e^3t - (1 / 6)e^2t + (7 / 2)e^-t\",\n \"D\": \"f(t) = [(- 4) / 3]e^-2t - (1 / 6)e^t - (7 / 2)e^3t\",\n \"E\": \"f(t) = [(- 4) / 3]e^2t + (1 / 6)e^t - (7 / 2)e^3t\",\n \"F\": \"f(t) = [(- 4) / 3]e^2t - (1 / 6)e^-t + (7 / 2)e^3t\",\n \"G\": \"f(t) = 4e^2t - (1 / 3)e^-t + (7 / 6)e^3t\",\n \"H\": \"f(t) = [(- 4) / 3]e^2t - (1 / 6)e^t + (7 / 2)e^3t\",\n \"I\": \"f(t) = [(- 4) / 3]e^2t + (1 / 6)e^-t + (7 / 2)e^3t\",\n \"J\": \"f(t) = (4 / 3)e^2t + (1 / 6)e^-t - (7 / 2)e^3t\"\n }\n}
input
question.json
{
"question": "Find the inverse Laplace transform of F(s) = [(2s^2 - 4) / {(s - 2)(s + 1)(s - 3)}]",
"options": {
"A": "f(t) = [(- 4) / 3]e^-2t - (1 / 6)e^2t + (7 / 2)e^-3t",
"B": "f(t) = [(- 4) / 3]e^2t - (1 / 6)e^-t - (7 / 2)e^3t",
"C": "f(t) = [(- 4) / 3]e^3t - (1 / 6)e^2t + (7 / 2)e^-t",
"D": "f(t) = [(- 4) / 3]e^-2t - (1 / 6)e^t - (7 / 2)e^3t",
"E": "f(t) = [(- 4) / 3]e^2t + (1 / 6)e^t - (7 / 2)e^3t",
"F": "f(t) = [(- 4) / 3]e^2t - (1 / 6)e^-t + (7 / 2)e^3t",
"G": "f(t) = 4e^2t - (1 / 3)e^-t + (7 / 6)e^3t",
"H": "f(t) = [(- 4) / 3]e^2t - (1 / 6)e^t + (7 / 2)e^3t",
"I": "f(t) = [(- 4) / 3]e^2t + (1 / 6)e^-t + (7 / 2)e^3t",
"J": "f(t) = (4 / 3)e^2t + (1 / 6)e^-t - (7 / 2)e^3t"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_056/ files.
Scored by a judge this pinned commit does not carry.
▸case_057Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A liquid is compressed isothermally inside a chamber. Obtain an expression for the total amount of work required if the compression process isquasistaticand is given by the equation In (V / V_0) = - A(p - p_0) where A, V_0 and p_0 are constants.\",\n \"options\": {\n \"A\": \"W = (AV_0) (1 / p_2 - 1 / p_1)\",\n \"B\": \"W = (A / V_0) (p_1 - p_2)\",\n \"C\": \"W = (A / V_0) ln(p_2 / p_1)\",\n \"D\": \"W= (AV / 2) (p_2^2 - p_1^2)\",\n \"E\": \"W = -A ln(V / V_0) (p_2 - p_1)\",\n \"F\": \"W= - (AV / 2) (p_1^2 - p_2^2)\",\n \"G\": \"W = A(V_0 / V) (p_2 - p_1)\",\n \"H\": \"W= - (AV / 2) (p_2^2 - p_1^2)\",\n \"I\": \"W = A(V_0 + V) (p_2 - p_1)\"\n }\n}
input
question.json
{
"question": "A liquid is compressed isothermally inside a chamber. Obtain an expression for the total amount of work required if the compression process isquasistaticand is given by the equation In (V / V_0) = - A(p - p_0) where A, V_0 and p_0 are constants.",
"options": {
"A": "W = (AV_0) (1 / p_2 - 1 / p_1)",
"B": "W = (A / V_0) (p_1 - p_2)",
"C": "W = (A / V_0) ln(p_2 / p_1)",
"D": "W= (AV / 2) (p_2^2 - p_1^2)",
"E": "W = -A ln(V / V_0) (p_2 - p_1)",
"F": "W= - (AV / 2) (p_1^2 - p_2^2)",
"G": "W = A(V_0 / V) (p_2 - p_1)",
"H": "W= - (AV / 2) (p_2^2 - p_1^2)",
"I": "W = A(V_0 + V) (p_2 - p_1)"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_057/ files.
Scored by a judge this pinned commit does not carry.
▸case_058Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Find the mass of air in a closed chamber measuring 35 ft × 20 ft × 10 ft, when the pressure is 17 lb/in^2 and the temperature is 75°F. Assume air to be an ideal gas.\",\n \"options\": {\n \"A\": \"550 lbm\",\n \"B\": \"650 lbm\",\n \"C\": \"601lbm\",\n \"D\": \"500lbm\",\n \"E\": \"800lbm\",\n \"F\": \"750 lbm\",\n \"G\": \"450 lbm\",\n \"H\": \"900 lbm\",\n \"I\": \"700lbm\",\n \"J\": \"850 lbm\"\n }\n}
input
question.json
{
"question": "Find the mass of air in a closed chamber measuring 35 ft × 20 ft × 10 ft, when the pressure is 17 lb/in^2 and the temperature is 75°F. Assume air to be an ideal gas.",
"options": {
"A": "550 lbm",
"B": "650 lbm",
"C": "601lbm",
"D": "500lbm",
"E": "800lbm",
"F": "750 lbm",
"G": "450 lbm",
"H": "900 lbm",
"I": "700lbm",
"J": "850 lbm"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_058/ files.
Scored by a judge this pinned commit does not carry.
▸case_059Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A vessel which contains gases A and B at 1atmand 40°F is connectedto another vessel with a different concentration of thegases at the same conditions of pressure and temperature. The vessels are connected by a 9 in. long tube ofinside diameter 1.5 in. (I)Determinethe steady state rate of transport of gas A betweenthe vessels if the concentration of A in one tank is 85 mole percent and that in the other is 7 mole percent. Transfer occurs by molecular diffusion and the diffusivity is D_AB = 1.75 ft^2 / hr. (II) The tanks are now at 5 and 70°F. Obtain the rate of heat transferby molecular transport. The following properties are given: \\\\rho = gas density = 0.07 lb / ft^3 C_p = heat capacity of gas = 0.30 Btu /lb°F \\\\alpha = thermal diffusivity = 1.75 ft^2 / hr\",\n \"options\": {\n \"A\": \"N_A = 8.0 × 10^-5 lb moles / hr, q = 0.065 Btu / hr\",\n \"B\": \"N_A = 7.0 × 10^-5 lb moles / hr, q = 0.02 Btu / hr\",\n \"C\": \"N_A = 5.0 × 10^-5 lb moles / hr, q = 0.03 Btu / hr\",\n \"D\": \"N_A = 6.07 × 10^-5 lb moles / hr, q = 0.04187 Btu / hr\",\n \"E\": \"N_A = 4.8 × 10^-5 lb moles / hr, q = 0.035 Btu / hr\",\n \"F\": \"N_A = 6.2 × 10^-5 lb moles / hr, q = 0.045 Btu / hr\",\n \"G\": \"N_A = 4.5 × 10^-5 lb moles / hr, q = 0.04 Btu / hr\",\n \"H\": \"N_A = 6.5 × 10^-5 lb moles / hr, q = 0.05 Btu / hr\",\n \"I\": \"N_A = 3.5 × 10^-5 lb moles / hr, q = 0.025 Btu / hr\",\n \"J\": \"N_A = 5.5 × 10^-5 lb moles / hr, q = 0.06 Btu / hr\"\n }\n}
input
question.json
{
"question": "A vessel which contains gases A and B at 1atmand 40°F is connectedto another vessel with a different concentration of thegases at the same conditions of pressure and temperature. The vessels are connected by a 9 in. long tube ofinside diameter 1.5 in. (I)Determinethe steady state rate of transport of gas A betweenthe vessels if the concentration of A in one tank is 85 mole percent and that in the other is 7 mole percent. Transfer occurs by molecular diffusion and the diffusivity is D_AB = 1.75 ft^2 / hr. (II) The tanks are now at 5 and 70°F. Obtain the rate of heat transferby molecular transport. The following properties are given: \\rho = gas density = 0.07 lb / ft^3 C_p = heat capacity of gas = 0.30 Btu /lb°F \\alpha = thermal diffusivity = 1.75 ft^2 / hr",
"options": {
"A": "N_A = 8.0 × 10^-5 lb moles / hr, q = 0.065 Btu / hr",
"B": "N_A = 7.0 × 10^-5 lb moles / hr, q = 0.02 Btu / hr",
"C": "N_A = 5.0 × 10^-5 lb moles / hr, q = 0.03 Btu / hr",
"D": "N_A = 6.07 × 10^-5 lb moles / hr, q = 0.04187 Btu / hr",
"E": "N_A = 4.8 × 10^-5 lb moles / hr, q = 0.035 Btu / hr",
"F": "N_A = 6.2 × 10^-5 lb moles / hr, q = 0.045 Btu / hr",
"G": "N_A = 4.5 × 10^-5 lb moles / hr, q = 0.04 Btu / hr",
"H": "N_A = 6.5 × 10^-5 lb moles / hr, q = 0.05 Btu / hr",
"I": "N_A = 3.5 × 10^-5 lb moles / hr, q = 0.025 Btu / hr",
"J": "N_A = 5.5 × 10^-5 lb moles / hr, q = 0.06 Btu / hr"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_059/ files.
Scored by a judge this pinned commit does not carry.
▸case_060Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A state has had a tremendous influx of retired people in recent years. There has been considerable concern among state health officials who foresee that many of the senior citizens will become victims of price gouging on certain medical supplies and services. In an attempt to curb such fraudulent sales practices, the state legislature has enacted a law prohibiting the sale of hearing aids by non-physicians. The measure provides, however, that all non-physician sellers who are presently engaged in the business of selling hearing aids will not be affected. Assume that after the statute goes into effect, a non-physician moves to the state and wants to open a business selling hearing aids. After being advised that the state law prohibits him from doing so, he brings suit challenging the constitutionality of the statute. The most likely result is that the state law will be declared\",\n \"options\": {\n \"A\": \"unconstitutional, because it violates the commerce clause, since Congress has plenary power to regulate any activity that has any appreciable effect on interstate commerce.\",\n \"B\": \"unconstitutional, because it creates an unfair trade barrier for non-physicians wanting to start a business in the state.\",\n \"C\": \"constitutional, as it aims to protect a vulnerable segment of the population from potential exploitation.\",\n \"D\": \"constitutional, because there is a rational basis for distinguishing between non-physicians are not so engaged.\",\n \"E\": \"unconstitutional, because it discriminates between non-physicians already in the business and those wanting to enter it.\",\n \"F\": \"unconstitutional, because it restricts the rights of non-physicians to conduct a legitimate business.\",\n \"G\": \"constitutional, because it falls under the state's power to regulate healthcare and related services.\",\n \"H\": \"constitutional, because it does not infringe upon the rights of non-physicians not already in the business of selling hearing aids.\",\n \"I\": \"constitutional, because a state has the power to regulate any phase of local business, even though such regulations may have some effect on interstate commerce, provided that Congress has not enacted legislation regarding the subject matter.\"\n }\n}
input
question.json
{
"question": "A state has had a tremendous influx of retired people in recent years. There has been considerable concern among state health officials who foresee that many of the senior citizens will become victims of price gouging on certain medical supplies and services. In an attempt to curb such fraudulent sales practices, the state legislature has enacted a law prohibiting the sale of hearing aids by non-physicians. The measure provides, however, that all non-physician sellers who are presently engaged in the business of selling hearing aids will not be affected. Assume that after the statute goes into effect, a non-physician moves to the state and wants to open a business selling hearing aids. After being advised that the state law prohibits him from doing so, he brings suit challenging the constitutionality of the statute. The most likely result is that the state law will be declared",
"options": {
"A": "unconstitutional, because it violates the commerce clause, since Congress has plenary power to regulate any activity that has any appreciable effect on interstate commerce.",
"B": "unconstitutional, because it creates an unfair trade barrier for non-physicians wanting to start a business in the state.",
"C": "constitutional, as it aims to protect a vulnerable segment of the population from potential exploitation.",
"D": "constitutional, because there is a rational basis for distinguishing between non-physicians are not so engaged.",
"E": "unconstitutional, because it discriminates between non-physicians already in the business and those wanting to enter it.",
"F": "unconstitutional, because it restricts the rights of non-physicians to conduct a legitimate business.",
"G": "constitutional, because it falls under the state's power to regulate healthcare and related services.",
"H": "constitutional, because it does not infringe upon the rights of non-physicians not already in the business of selling hearing aids.",
"I": "constitutional, because a state has the power to regulate any phase of local business, even though such regulations may have some effect on interstate commerce, provided that Congress has not enacted legislation regarding the subject matter."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_060/ files.
Scored by a judge this pinned commit does not carry.
▸case_061Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Calculate the width of a silent chain of 1 in. pitch, which delivers 25 hp. The driving sprocket, having 25 teeth, rotates at 400 rpm. Chain Pitch, in. Allowable Tension lb/in. 3/8 75 1/2 100 5/8 125 3/4 150 1 205 1(1/4) 265 1(1/2) 335 2 600 Table 1 Load handling capacities of silent chains\",\n \"options\": {\n \"A\": \"4.0 in\",\n \"B\": \"3.25 in\",\n \"C\": \"5.75 in\",\n \"D\": \"5.2 in\",\n \"E\": \"6.0 in\",\n \"F\": \"3.75 in\",\n \"G\": \"2.5 in\",\n \"H\": \"7.0 in\",\n \"I\": \"4.83 in\",\n \"J\": \"4.5 in\"\n }\n}
input
question.json
{
"question": "Calculate the width of a silent chain of 1 in. pitch, which delivers 25 hp. The driving sprocket, having 25 teeth, rotates at 400 rpm. Chain Pitch, in. Allowable Tension lb/in. 3/8 75 1/2 100 5/8 125 3/4 150 1 205 1(1/4) 265 1(1/2) 335 2 600 Table 1 Load handling capacities of silent chains",
"options": {
"A": "4.0 in",
"B": "3.25 in",
"C": "5.75 in",
"D": "5.2 in",
"E": "6.0 in",
"F": "3.75 in",
"G": "2.5 in",
"H": "7.0 in",
"I": "4.83 in",
"J": "4.5 in"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_061/ files.
Scored by a judge this pinned commit does not carry.
▸case_062Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Is extra-territorial abduction a ground for refusal of otherwise lawful jurisdiction by English courts?\",\n \"options\": {\n \"A\": \"Such abduction constitutes abuse of process and leads to a stay of proceedings\",\n \"B\": \"Such abduction gives rise to the abductor's liability but does not lead to a stay of proceedings\",\n \"C\": \"The stay of proceedings under such circumstances would prevent the administration of justice in respect of international criminals\",\n \"D\": \"The courts may, but will seldom, stay proceedings in cases of extra-territorial abduction\"\n }\n}
input
question.json
{
"question": "Is extra-territorial abduction a ground for refusal of otherwise lawful jurisdiction by English courts?",
"options": {
"A": "Such abduction constitutes abuse of process and leads to a stay of proceedings",
"B": "Such abduction gives rise to the abductor's liability but does not lead to a stay of proceedings",
"C": "The stay of proceedings under such circumstances would prevent the administration of justice in respect of international criminals",
"D": "The courts may, but will seldom, stay proceedings in cases of extra-territorial abduction"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_062/ files.
Scored by a judge this pinned commit does not carry.
▸case_063Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A newly elected mayor was charged with tax fraud. The prosecutor served a trial subpoena directing the mayor's attorney to produce private bank records that the mayor delivered to the attorney. The attorney refused the request. The prosecutor in a hearing then sought to compel compliance with the subpoena. The court should\",\n \"options\": {\n \"A\": \"compel production, because it would not violate the attorney-client privilege.\",\n \"B\": \"compel production, because the mayor has waived his rights to attorney-client privilege by delivering the records to his attorney.\",\n \"C\": \"not compel production, because it would violate the mayor's privilege against self- incrimination.\",\n \"D\": \"not compel production, because it would violate the attorney-client privilege.\",\n \"E\": \"compel production, because the attorney-client privilege does not extend to financial records.\",\n \"F\": \"not compel production, because the subpoena was not properly served.\",\n \"G\": \"not compel production, because the mayor's financial records are not relevant to the case.\",\n \"H\": \"compel production, because the mayor's financial records could be crucial evidence in the trial.\",\n \"I\": \"not compel production, because it would infringe on the mayor's right to privacy.\",\n \"J\": \"compel production, because there is no element of confidentiality.\"\n }\n}
input
question.json
{
"question": "A newly elected mayor was charged with tax fraud. The prosecutor served a trial subpoena directing the mayor's attorney to produce private bank records that the mayor delivered to the attorney. The attorney refused the request. The prosecutor in a hearing then sought to compel compliance with the subpoena. The court should",
"options": {
"A": "compel production, because it would not violate the attorney-client privilege.",
"B": "compel production, because the mayor has waived his rights to attorney-client privilege by delivering the records to his attorney.",
"C": "not compel production, because it would violate the mayor's privilege against self- incrimination.",
"D": "not compel production, because it would violate the attorney-client privilege.",
"E": "compel production, because the attorney-client privilege does not extend to financial records.",
"F": "not compel production, because the subpoena was not properly served.",
"G": "not compel production, because the mayor's financial records are not relevant to the case.",
"H": "compel production, because the mayor's financial records could be crucial evidence in the trial.",
"I": "not compel production, because it would infringe on the mayor's right to privacy.",
"J": "compel production, because there is no element of confidentiality."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_063/ files.
Scored by a judge this pinned commit does not carry.
▸case_064Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A defendant wished to see his high school basketball team win the state championship. During an important game, the defendant pulled out a gun and shot at the leg of a key player on the opposing team. The defendant intended only to inflict a slight wound so that the opposing player would be unable to complete the game. When the defendant fired the shot, he unintentionally hit a player on his own high school team in the chest, killing him instantly. What is the most serious crime that the defendant can be convicted of? \",\n \"options\": {\n \"A\": \"Justifiable homicide.\",\n \"B\": \"Reckless endangerment.\",\n \"C\": \"Battery.\",\n \"D\": \"Attempted murder.\",\n \"E\": \"Felonious assault.\",\n \"F\": \"Assault with a deadly weapon.\",\n \"G\": \"Voluntary manslaughter.\",\n \"H\": \"Negligent homicide.\",\n \"I\": \"Involuntary manslaughter.\",\n \"J\": \"Murder.\"\n }\n}
input
question.json
{
"question": "A defendant wished to see his high school basketball team win the state championship. During an important game, the defendant pulled out a gun and shot at the leg of a key player on the opposing team. The defendant intended only to inflict a slight wound so that the opposing player would be unable to complete the game. When the defendant fired the shot, he unintentionally hit a player on his own high school team in the chest, killing him instantly. What is the most serious crime that the defendant can be convicted of? ",
"options": {
"A": "Justifiable homicide.",
"B": "Reckless endangerment.",
"C": "Battery.",
"D": "Attempted murder.",
"E": "Felonious assault.",
"F": "Assault with a deadly weapon.",
"G": "Voluntary manslaughter.",
"H": "Negligent homicide.",
"I": "Involuntary manslaughter.",
"J": "Murder."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_064/ files.
Scored by a judge this pinned commit does not carry.
▸case_065Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A lumber mill contracted to deliver one thousand8-foot sheets of plywood to a home improvement store on the first of each month for 10 consecutive months starting June 1. The June, July, and August shipments were delivered on time and duly paid for. Then, on August 16, the lumber mill informed the store that the mill would not be able to meet the September 1 delivery date, because its lumber reserve had been destroyed by a forest fire. The mill then asked the store to excuse the mill from further performance. The store refused and demanded that the remaining shipments be delivered on time. When the September shipment failed to arrive, the store immediately brought suit for breach of contract. How would the court hold?\",\n \"options\": {\n \"A\": \"Judgment for the mill, because their performance would be discharged by impossibility.\",\n \"B\": \"Judgment for the store, because the mill's duties of performance would not be excused.\",\n \"C\": \"Judgment for the mill, because the forest fire caused an unforeseeable hindrance in performance.\",\n \"D\": \"Judgment for the mill, because their performance would be discharged by frustration of purpose.\",\n \"E\": \"Judgment for the mill, because the forest fire was an act of God and beyond their control.\",\n \"F\": \"Judgment for the mill, because they had informed the store about the issue in advance.\"\n }\n}
input
question.json
{
"question": "A lumber mill contracted to deliver one thousand8-foot sheets of plywood to a home improvement store on the first of each month for 10 consecutive months starting June 1. The June, July, and August shipments were delivered on time and duly paid for. Then, on August 16, the lumber mill informed the store that the mill would not be able to meet the September 1 delivery date, because its lumber reserve had been destroyed by a forest fire. The mill then asked the store to excuse the mill from further performance. The store refused and demanded that the remaining shipments be delivered on time. When the September shipment failed to arrive, the store immediately brought suit for breach of contract. How would the court hold?",
"options": {
"A": "Judgment for the mill, because their performance would be discharged by impossibility.",
"B": "Judgment for the store, because the mill's duties of performance would not be excused.",
"C": "Judgment for the mill, because the forest fire caused an unforeseeable hindrance in performance.",
"D": "Judgment for the mill, because their performance would be discharged by frustration of purpose.",
"E": "Judgment for the mill, because the forest fire was an act of God and beyond their control.",
"F": "Judgment for the mill, because they had informed the store about the issue in advance."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_065/ files.
Scored by a judge this pinned commit does not carry.
▸case_066Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A landlord was the owner of a vacant warehouse. On May 1, the landlord leased the warehouse to the tenant for a term of five years with a renewable option. The leasehold agreement provided in part:\\\"The tenant hereby acknowledges that certain areas of the leased premises are in a state of disrepair and unsafe for the conduct of business. Nonetheless, the tenant agrees to assume full responsibility for the necessary repairs. Furthermore, the tenant agrees to indemnify the landlord for any loss resulting from the condition of the said premises. \\\"Under the terms of the lease, the landlord delivered possession of the warehouse to the tenant on May 2. On that date, an employee of the tenant was working in the warehouse. He was moving some equipment into the warehouse when a section of the ceiling collapsed and fell on him. At the time the landlord and the tenant entered into their lease, only the landlord was aware that the ceiling was defective and needed to be repaired. Although the landlord could have remedied the defect before delivering possession to the tenant, the landlord didn't perform any repair work on the ceiling. The employee initiated suit against the landlord to recover damages resulting from his injury. There are no applicable statutes. If the employee prevails it will be because a\",\n \"options\": {\n \"A\": \"landowner's duty of care to third parties cannot be shifted to a tenant by the tenns of a lease.\",\n \"B\": \"tenant is responsible for all injuries occurring on the property once possession has been transferred.\",\n \"C\": \"tenant cannot waive the implied warranty of commercial habitability.\",\n \"D\": \"landlord is absolved of all responsibilities once a lease agreement is signed.\",\n \"E\": \"landlord's failure to repair the defect is not relevant to the case.\",\n \"F\": \"covenant to indemnify by a tenant in favor of a land owner is against public policy.\",\n \"G\": \"tenant's employee was not protected under the leasehold agreement.\",\n \"H\": \"landowner is strictly liable for injuries occurring on his property.\",\n \"I\": \"landlord's duty of care to a tenant can be transferred through a lease agreement.\",\n \"J\": \"tenant's acknowledgment of the state of disrepair absolves the landlord of any liability.\"\n }\n}
input
question.json
{
"question": "A landlord was the owner of a vacant warehouse. On May 1, the landlord leased the warehouse to the tenant for a term of five years with a renewable option. The leasehold agreement provided in part:\"The tenant hereby acknowledges that certain areas of the leased premises are in a state of disrepair and unsafe for the conduct of business. Nonetheless, the tenant agrees to assume full responsibility for the necessary repairs. Furthermore, the tenant agrees to indemnify the landlord for any loss resulting from the condition of the said premises. \"Under the terms of the lease, the landlord delivered possession of the warehouse to the tenant on May 2. On that date, an employee of the tenant was working in the warehouse. He was moving some equipment into the warehouse when a section of the ceiling collapsed and fell on him. At the time the landlord and the tenant entered into their lease, only the landlord was aware that the ceiling was defective and needed to be repaired. Although the landlord could have remedied the defect before delivering possession to the tenant, the landlord didn't perform any repair work on the ceiling. The employee initiated suit against the landlord to recover damages resulting from his injury. There are no applicable statutes. If the employee prevails it will be because a",
"options": {
"A": "landowner's duty of care to third parties cannot be shifted to a tenant by the tenns of a lease.",
"B": "tenant is responsible for all injuries occurring on the property once possession has been transferred.",
"C": "tenant cannot waive the implied warranty of commercial habitability.",
"D": "landlord is absolved of all responsibilities once a lease agreement is signed.",
"E": "landlord's failure to repair the defect is not relevant to the case.",
"F": "covenant to indemnify by a tenant in favor of a land owner is against public policy.",
"G": "tenant's employee was not protected under the leasehold agreement.",
"H": "landowner is strictly liable for injuries occurring on his property.",
"I": "landlord's duty of care to a tenant can be transferred through a lease agreement.",
"J": "tenant's acknowledgment of the state of disrepair absolves the landlord of any liability."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_066/ files.
Scored by a judge this pinned commit does not carry.
▸case_067Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Air is flowing through a nozzle with a 90% efficiency and an expansion rate of 2.0 Kg/s. The inlet conditions are P = 550kPa and T = 680°K. If the exit pressure is 120kPa calculate the actual exit area, velocity and enthalpy. Assume K = 1.4.\",\n \"options\": {\n \"A\": \"Exit area = 2.37 × 10^-3m^2, Velocity = 550 m/s, Enthalpy = 400 KJ/kg\",\n \"B\": \"Exit area = 5.37 × 10^-3m^2, Velocity = 850 m/s, Enthalpy = 550 KJ/kg\",\n \"C\": \"Exit area = 6.37 × 10^-3m^2, Velocity = 900 m/s, Enthalpy = 610 KJ/kg\",\n \"D\": \"Exit area = 3.37 × 10^-3m^2, Velocity = 658 m/s, Enthalpy = 465.8 KJ/kg\",\n \"E\": \"Exit area = 4.97 × 10^-3m^2, Velocity = 800 m/s, Enthalpy = 480 KJ/kg\",\n \"F\": \"Exit area = 3.87 × 10^-3m^2, Velocity = 620 m/s, Enthalpy = 560 KJ/kg\",\n \"G\": \"Exit area = 2.97 × 10^-3m^2, Velocity = 700 m/s, Enthalpy = 520 KJ/kg\",\n \"H\": \"Exit area = 1.37 × 10^-3m^2, Velocity = 600 m/s, Enthalpy = 580 KJ/kg\",\n \"I\": \"Exit area = 4.37 × 10^-3m^2, Velocity = 750 m/s, Enthalpy = 500 KJ/kg\",\n \"J\": \"Exit area = 2.77 × 10^-3m^2, Velocity = 780 m/s, Enthalpy = 530 KJ/kg\"\n }\n}
input
question.json
{
"question": "Air is flowing through a nozzle with a 90% efficiency and an expansion rate of 2.0 Kg/s. The inlet conditions are P = 550kPa and T = 680°K. If the exit pressure is 120kPa calculate the actual exit area, velocity and enthalpy. Assume K = 1.4.",
"options": {
"A": "Exit area = 2.37 × 10^-3m^2, Velocity = 550 m/s, Enthalpy = 400 KJ/kg",
"B": "Exit area = 5.37 × 10^-3m^2, Velocity = 850 m/s, Enthalpy = 550 KJ/kg",
"C": "Exit area = 6.37 × 10^-3m^2, Velocity = 900 m/s, Enthalpy = 610 KJ/kg",
"D": "Exit area = 3.37 × 10^-3m^2, Velocity = 658 m/s, Enthalpy = 465.8 KJ/kg",
"E": "Exit area = 4.97 × 10^-3m^2, Velocity = 800 m/s, Enthalpy = 480 KJ/kg",
"F": "Exit area = 3.87 × 10^-3m^2, Velocity = 620 m/s, Enthalpy = 560 KJ/kg",
"G": "Exit area = 2.97 × 10^-3m^2, Velocity = 700 m/s, Enthalpy = 520 KJ/kg",
"H": "Exit area = 1.37 × 10^-3m^2, Velocity = 600 m/s, Enthalpy = 580 KJ/kg",
"I": "Exit area = 4.37 × 10^-3m^2, Velocity = 750 m/s, Enthalpy = 500 KJ/kg",
"J": "Exit area = 2.77 × 10^-3m^2, Velocity = 780 m/s, Enthalpy = 530 KJ/kg"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_067/ files.
Scored by a judge this pinned commit does not carry.
▸case_068Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"The P, v, T relation of a real gas is represented with reasonable accuracy by the relation v = (RT / P) + b - (a / RT) where a and b are constants. For this gas find the change in enthalpy and entropy along an isothermal path between pressures P_1 and P_2.\",\n \"options\": {\n \"A\": \"(h_2 - h_1)_T = [b - (2a / RT)] (P_1 - P_2), (s_1 - s_2)_T = Rln(P_2 / P_1) - [{a(P_2 - P_1)} / (RT^2)]\",\n \"B\": \"(h_2 - h_1)_T = [b - (3a / RT)] (P_2 - P_1), (s_1 - s_2)_T = - Rln(P_2 / P_1) - [{a(P_2 - P_1)} / (RT^3)]\",\n \"C\": \"(h_2 - h_1)_T = [b - (a / RT)] (P_2 - P_1), (s_1 - s_2)_T = Rln(P_2 / P_1) + [{a(P_2 - P_1)} / (RT^2)]\",\n \"D\": \"(h_2 - h_1)_T = [b - (2a / RT)] (P_2 - P_1), (s_1 - s_2)_T = - Rln(P_1 / P_2) - [{a(P_1 - P_2)} / (RT^2)]\",\n \"E\": \"(h_2 - h_1)_T = [b + (2a / RT)] (P_1 - P_2), (s_1 - s_2)_T = - Rln(P_1 / P_2) + [{a(P_1 - P_2)} / (RT^2)]\",\n \"F\": \"(h_2 - h_1)_T = [b - (2a / RT)] (P_1 - P_2), (s_1 - s_2)_T = - Rln(P_1 / P_2) + [{a(P_1 - P_2)} / (RT^2)]\",\n \"G\": \"(h_2 - h_1)_T = [b + (2a / RT)] (P_2 - P_1), (s_1 - s_2)_T = - Rln(P_2 / P_1) + [{a(P_2 - P_1)} / (RT^2)]\",\n \"H\": \"(h_2 - h_1)_T = [b + (2a / RT)] (P_2 - P_1), (s_1 - s_2)_T = Rln(P_1 / P_2) + [{a(P_1 - P_2)} / (RT^2)]\",\n \"I\": \"(h_2 - h_1)_T = [b - (2a / RT)] (P_2 - P_1), (s_1 - s_2)_T = - Rln(P_2 / P_1) - [{a(P_2 - P_1)} / (RT^2)]\",\n \"J\": \"(h_2 - h_1)_T = [b + (a / RT)] (P_2 - P_1), (s_1 - s_2)_T = - Rln(P_1 / P_2) - [{a(P_1 - P_2)} / (RT^2)]\"\n }\n}
input
question.json
{
"question": "The P, v, T relation of a real gas is represented with reasonable accuracy by the relation v = (RT / P) + b - (a / RT) where a and b are constants. For this gas find the change in enthalpy and entropy along an isothermal path between pressures P_1 and P_2.",
"options": {
"A": "(h_2 - h_1)_T = [b - (2a / RT)] (P_1 - P_2), (s_1 - s_2)_T = Rln(P_2 / P_1) - [{a(P_2 - P_1)} / (RT^2)]",
"B": "(h_2 - h_1)_T = [b - (3a / RT)] (P_2 - P_1), (s_1 - s_2)_T = - Rln(P_2 / P_1) - [{a(P_2 - P_1)} / (RT^3)]",
"C": "(h_2 - h_1)_T = [b - (a / RT)] (P_2 - P_1), (s_1 - s_2)_T = Rln(P_2 / P_1) + [{a(P_2 - P_1)} / (RT^2)]",
"D": "(h_2 - h_1)_T = [b - (2a / RT)] (P_2 - P_1), (s_1 - s_2)_T = - Rln(P_1 / P_2) - [{a(P_1 - P_2)} / (RT^2)]",
"E": "(h_2 - h_1)_T = [b + (2a / RT)] (P_1 - P_2), (s_1 - s_2)_T = - Rln(P_1 / P_2) + [{a(P_1 - P_2)} / (RT^2)]",
"F": "(h_2 - h_1)_T = [b - (2a / RT)] (P_1 - P_2), (s_1 - s_2)_T = - Rln(P_1 / P_2) + [{a(P_1 - P_2)} / (RT^2)]",
"G": "(h_2 - h_1)_T = [b + (2a / RT)] (P_2 - P_1), (s_1 - s_2)_T = - Rln(P_2 / P_1) + [{a(P_2 - P_1)} / (RT^2)]",
"H": "(h_2 - h_1)_T = [b + (2a / RT)] (P_2 - P_1), (s_1 - s_2)_T = Rln(P_1 / P_2) + [{a(P_1 - P_2)} / (RT^2)]",
"I": "(h_2 - h_1)_T = [b - (2a / RT)] (P_2 - P_1), (s_1 - s_2)_T = - Rln(P_2 / P_1) - [{a(P_2 - P_1)} / (RT^2)]",
"J": "(h_2 - h_1)_T = [b + (a / RT)] (P_2 - P_1), (s_1 - s_2)_T = - Rln(P_1 / P_2) - [{a(P_1 - P_2)} / (RT^2)]"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_068/ files.
Scored by a judge this pinned commit does not carry.
▸case_069Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Water film boiling occurs on top of a horizontal plate. The wateris at a pressure of 1 atm. and the surface temperature ofthe plate is 554°C. Calculate the heat flux given off by waterassuming a characteristic length (L_c) of 1 cm.\",\n \"options\": {\n \"A\": \"70,830 W/m^2 or 7.083 W/cm^2\",\n \"B\": \"65,830 W/m^2 or 6.583 W/cm^2\",\n \"C\": \"45,830 W/m^2 or 4.583 W/cm^2\",\n \"D\": \"80,000 W/m^2 or 8.000 W/cm^2\",\n \"E\": \"40,000 W/m^2 or 4.000 W/cm^2\",\n \"F\": \"48,500 W/m^2 or 4.850 W/cm^2\",\n \"G\": \"75,000 W/m^2 or 7.500 W/cm^2\",\n \"H\": \"68,500 W/m^2 or 6.850 W/cm^2\",\n \"I\": \"52,000 W/m^2 or 5.200 W/cm^2\",\n \"J\": \"59,830 W/m^2 or 5.983 W/cm^2\"\n }\n}
input
question.json
{
"question": "Water film boiling occurs on top of a horizontal plate. The wateris at a pressure of 1 atm. and the surface temperature ofthe plate is 554°C. Calculate the heat flux given off by waterassuming a characteristic length (L_c) of 1 cm.",
"options": {
"A": "70,830 W/m^2 or 7.083 W/cm^2",
"B": "65,830 W/m^2 or 6.583 W/cm^2",
"C": "45,830 W/m^2 or 4.583 W/cm^2",
"D": "80,000 W/m^2 or 8.000 W/cm^2",
"E": "40,000 W/m^2 or 4.000 W/cm^2",
"F": "48,500 W/m^2 or 4.850 W/cm^2",
"G": "75,000 W/m^2 or 7.500 W/cm^2",
"H": "68,500 W/m^2 or 6.850 W/cm^2",
"I": "52,000 W/m^2 or 5.200 W/cm^2",
"J": "59,830 W/m^2 or 5.983 W/cm^2"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_069/ files.
Scored by a judge this pinned commit does not carry.
▸case_070Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A football team entered into a 10-year lease with a city for use of the city's athletic stadium. Five years into the lease, the team threatened to leave the stadium and move to another city. The city sued the team in federal court, seeking a permanent injunction to prevent the team from breaching its lease and leaving. In its answer, the team included a counterclaim seeking $10 million in damages for losses caused by the city's alleged failure to properly maintain the stadium, as the lease required. The team demanded a jury trial on the counterclaim. The city moved to try its claim for a permanent injunction before the trial on the team's counterclaim. The team objected and moved that the jury trial of its counterclaim be held before the trial of the city's injunction claim. How should the court rule on the parties' motions?\",\n \"options\": {\n \"A\": \"The court should first hold a nonjury trial of the city's claim without giving binding effect to its findings or conclusions in the later jury trial of the team's counterclaim.\",\n \"B\": \"The court should first hold a jury trial of the team's counterclaim, and its findings should be binding in the later nonjury trial of the city's claim.\",\n \"C\": \"The court should first hold a nonjury trial of the team's counterclaim, and then a jury trial of the city's claim.\",\n \"D\": \"The court should schedule a jury trial of both the city's claim and the team's counterclaim.\",\n \"E\": \"The court should hold a simultaneous jury trial of the team's counterclaim and nonjury trial of the city's claim.\",\n \"F\": \"The court should first hold a jury trial of the city's claim, and then a nonjury trial of the team's counterclaim.\",\n \"G\": \"The court should first hold a jury trial of the team's counterclaim, and then a nonjury trial of the issues remaining in the city's claim.\",\n \"H\": \"The court should schedule a nonjury trial of both the city's claim and the team's counterclaim.\",\n \"I\": \"The court should first hold a nonjury trial of the city's claim, and then a jury trial of the issues remaining in the team's counterclaim.\",\n \"J\": \"The court should first hold a nonjury trial of the city's claim, and its findings should be binding in the later jury trial of the team's counterclaim.\"\n }\n}
input
question.json
{
"question": "A football team entered into a 10-year lease with a city for use of the city's athletic stadium. Five years into the lease, the team threatened to leave the stadium and move to another city. The city sued the team in federal court, seeking a permanent injunction to prevent the team from breaching its lease and leaving. In its answer, the team included a counterclaim seeking $10 million in damages for losses caused by the city's alleged failure to properly maintain the stadium, as the lease required. The team demanded a jury trial on the counterclaim. The city moved to try its claim for a permanent injunction before the trial on the team's counterclaim. The team objected and moved that the jury trial of its counterclaim be held before the trial of the city's injunction claim. How should the court rule on the parties' motions?",
"options": {
"A": "The court should first hold a nonjury trial of the city's claim without giving binding effect to its findings or conclusions in the later jury trial of the team's counterclaim.",
"B": "The court should first hold a jury trial of the team's counterclaim, and its findings should be binding in the later nonjury trial of the city's claim.",
"C": "The court should first hold a nonjury trial of the team's counterclaim, and then a jury trial of the city's claim.",
"D": "The court should schedule a jury trial of both the city's claim and the team's counterclaim.",
"E": "The court should hold a simultaneous jury trial of the team's counterclaim and nonjury trial of the city's claim.",
"F": "The court should first hold a jury trial of the city's claim, and then a nonjury trial of the team's counterclaim.",
"G": "The court should first hold a jury trial of the team's counterclaim, and then a nonjury trial of the issues remaining in the city's claim.",
"H": "The court should schedule a nonjury trial of both the city's claim and the team's counterclaim.",
"I": "The court should first hold a nonjury trial of the city's claim, and then a jury trial of the issues remaining in the team's counterclaim.",
"J": "The court should first hold a nonjury trial of the city's claim, and its findings should be binding in the later jury trial of the team's counterclaim."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_070/ files.
Scored by a judge this pinned commit does not carry.
▸case_071Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"What is passive personality jurisdiction? \",\n \"options\": {\n \"A\": \"It is jurisdiction based on the relationship between the victim and the offender\",\n \"B\": \"It is jurisdiction based on the country of residence of the offender\",\n \"C\": \"It is jurisdiction based on where the offence was committed\",\n \"D\": \"It is jurisdiction based on the country where the legal person was Registered\",\n \"E\": \"It is jurisdiction based on the international law applicable to the crime\",\n \"F\": \"It is jurisdiction based on the nationality of the offender\",\n \"G\": \"It is jurisdiction based on the nationality of the victims\",\n \"H\": \"It is jurisdiction based on the nationality of the witness\",\n \"I\": \"It is jurisdiction based on the severity of the offence\",\n \"J\": \"It is jurisdiction based on the country where the crime was planned\"\n }\n}
input
question.json
{
"question": "What is passive personality jurisdiction? ",
"options": {
"A": "It is jurisdiction based on the relationship between the victim and the offender",
"B": "It is jurisdiction based on the country of residence of the offender",
"C": "It is jurisdiction based on where the offence was committed",
"D": "It is jurisdiction based on the country where the legal person was Registered",
"E": "It is jurisdiction based on the international law applicable to the crime",
"F": "It is jurisdiction based on the nationality of the offender",
"G": "It is jurisdiction based on the nationality of the victims",
"H": "It is jurisdiction based on the nationality of the witness",
"I": "It is jurisdiction based on the severity of the offence",
"J": "It is jurisdiction based on the country where the crime was planned"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_071/ files.
Scored by a judge this pinned commit does not carry.
▸case_072Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Givenf(t) = ^+\\\\infty\\\\int_-\\\\infty h(\\\\sigma)\\\\textphi(t - \\\\sigma)d\\\\sigma determine the power spectral density of f(t) in terms of the power spectral density of \\\\textphi(t) and the Fourier transform of h(t).\",\n \"options\": {\n \"A\": \"G_f(\\\\omega) = \\u000bertH(\\\\omega)\\u000bert^2 G_\\textphi(\\\\omega)\",\n \"B\": \"\\\\( G_f(\\\\omega) = \\\\frac{G_{\\\\textphi}(\\\\omega)}{|H(\\\\omega)|^2} \\\\)\",\n \"C\": \"\\\\( G_f(\\\\omega) = H(\\\\omega) G_{\\\\textphi}(\\\\omega)^2 \\\\)\",\n \"D\": \"\\\\( G_f(\\\\omega) = |H(\\\\omega)|^2 / G_{\\\\textphi}(\\\\omega) \\\\)\",\n \"E\": \"\\\\( G_f(\\\\omega) = \\\\frac{1}{|H(\\\\omega)|^2} G_{\\\\textphi}(\\\\omega) \\\\)\",\n \"F\": \"\\\\( G_f(\\\\omega) = |H(\\\\omega)| G_{\\\\textphi}(\\\\omega)^2 \\\\)\",\n \"G\": \"G_f(\\\\omega) = \\u000bertH(\\\\omega)\\u000bert^2 / G_\\textphi(\\\\omega)\",\n \"H\": \"G_f(\\\\omega) = H(\\\\omega)^2 G_\\textphi(\\\\omega)\",\n \"I\": \"\\\\( G_f(\\\\omega) = \\\\frac{G_{\\\\textphi}(\\\\omega)}{H(\\\\omega)} \\\\)\",\n \"J\": \"G_f(\\\\omega) = \\u000bertH(\\\\omega)\\u000bert G_\\textphi(t)\"\n }\n}
input
question.json
{
"question": "Givenf(t) = ^+\\infty\\int_-\\infty h(\\sigma)\\textphi(t - \\sigma)d\\sigma determine the power spectral density of f(t) in terms of the power spectral density of \\textphi(t) and the Fourier transform of h(t).",
"options": {
"A": "G_f(\\omega) = \u000bertH(\\omega)\u000bert^2 G_\textphi(\\omega)",
"B": "\\( G_f(\\omega) = \\frac{G_{\\textphi}(\\omega)}{|H(\\omega)|^2} \\)",
"C": "\\( G_f(\\omega) = H(\\omega) G_{\\textphi}(\\omega)^2 \\)",
"D": "\\( G_f(\\omega) = |H(\\omega)|^2 / G_{\\textphi}(\\omega) \\)",
"E": "\\( G_f(\\omega) = \\frac{1}{|H(\\omega)|^2} G_{\\textphi}(\\omega) \\)",
"F": "\\( G_f(\\omega) = |H(\\omega)| G_{\\textphi}(\\omega)^2 \\)",
"G": "G_f(\\omega) = \u000bertH(\\omega)\u000bert^2 / G_\textphi(\\omega)",
"H": "G_f(\\omega) = H(\\omega)^2 G_\textphi(\\omega)",
"I": "\\( G_f(\\omega) = \\frac{G_{\\textphi}(\\omega)}{H(\\omega)} \\)",
"J": "G_f(\\omega) = \u000bertH(\\omega)\u000bert G_\textphi(t)"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_072/ files.
Scored by a judge this pinned commit does not carry.
▸case_073Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A defendant was driving his new sports car at a high rate of speed on a busy city street in the middle of the day. A pedestrian was struck by the defendant's car as she was crossing the street and was seriously injured. A criminal complaint was filed against the defendant and, at trial, he testified that he did not intend to injure anyone, but admitted that he was indifferent as to the consequences of driving recklessly. Based on the defendant's actions and testimony, he should be found guilty of\",\n \"options\": {\n \"A\": \"attempted murder.\",\n \"B\": \"attempted manslaughter.\",\n \"C\": \"assault with the intent to cause serious bodily injury.\",\n \"D\": \"battery.\"\n }\n}
input
question.json
{
"question": "A defendant was driving his new sports car at a high rate of speed on a busy city street in the middle of the day. A pedestrian was struck by the defendant's car as she was crossing the street and was seriously injured. A criminal complaint was filed against the defendant and, at trial, he testified that he did not intend to injure anyone, but admitted that he was indifferent as to the consequences of driving recklessly. Based on the defendant's actions and testimony, he should be found guilty of",
"options": {
"A": "attempted murder.",
"B": "attempted manslaughter.",
"C": "assault with the intent to cause serious bodily injury.",
"D": "battery."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_073/ files.
Scored by a judge this pinned commit does not carry.
▸case_074Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Given the following data for diamond and graphite at 25°C and 1atm Diamond Graphite Units g 1.233 0 Btu / lb mol - °R v 0.00456 0.00712 ft^3/lbm \\\\beta_T 0.16 × 10^-6 3.0 × 10^-6 atm^-1 determine the pressure that is needed to make diamonds from graphite at 25°C.\",\n \"options\": {\n \"A\": \"25,000 atm\",\n \"B\": \"12,000 atm\",\n \"C\": \"15,500atm\",\n \"D\": \"8,500 atm\",\n \"E\": \"5,000atm\",\n \"F\": \"18,000 atm\",\n \"G\": \"7,500 atm\",\n \"H\": \"20,000atm\",\n \"I\": \"30,000 atm\",\n \"J\": \"10,000atm\"\n }\n}
input
question.json
{
"question": "Given the following data for diamond and graphite at 25°C and 1atm Diamond Graphite Units g 1.233 0 Btu / lb mol - °R v 0.00456 0.00712 ft^3/lbm \\beta_T 0.16 × 10^-6 3.0 × 10^-6 atm^-1 determine the pressure that is needed to make diamonds from graphite at 25°C.",
"options": {
"A": "25,000 atm",
"B": "12,000 atm",
"C": "15,500atm",
"D": "8,500 atm",
"E": "5,000atm",
"F": "18,000 atm",
"G": "7,500 atm",
"H": "20,000atm",
"I": "30,000 atm",
"J": "10,000atm"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_074/ files.
Scored by a judge this pinned commit does not carry.
▸case_075Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A sinusoidal signal is analog signal, because\",\n \"options\": {\n \"A\": \"it is a square wave signal\",\n \"B\": \"it has a constant frequency\",\n \"C\": \"it is positive for one half cycle\",\n \"D\": \"it can only have a single value at a time\",\n \"E\": \"it has a sinusoidal waveform\",\n \"F\": \"it can only have two values: positive and negative\",\n \"G\": \"it can have a number of values between the negative and positive peaks\",\n \"H\": \"it has positive as well as negative values\",\n \"I\": \"it is a digital signal\",\n \"J\": \"it is negative for one half cycle\"\n }\n}
input
question.json
{
"question": "A sinusoidal signal is analog signal, because",
"options": {
"A": "it is a square wave signal",
"B": "it has a constant frequency",
"C": "it is positive for one half cycle",
"D": "it can only have a single value at a time",
"E": "it has a sinusoidal waveform",
"F": "it can only have two values: positive and negative",
"G": "it can have a number of values between the negative and positive peaks",
"H": "it has positive as well as negative values",
"I": "it is a digital signal",
"J": "it is negative for one half cycle"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_075/ files.
Scored by a judge this pinned commit does not carry.
▸case_076Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Propane burns in dry 125 percent theoretical air (21% O_2, 79% N_2) and the reaction is given by: C_3H_8 + 5O_2 + 18.8N_2 \\\\rightarrow 3CO_2 + 4H_2O + 18.8N_2 Calculate the dew point and the mole fractions of the combustionproducts when the air is at 1atmpressure.\",\n \"options\": {\n \"A\": \"Dew Point: 133F, Mole Fractions: CO2: 0.097, H2O: 0.123, O2: 0.042, N2: 0.738\",\n \"B\": \"Dew Point: 138F, Mole Fractions: CO2: 0.094, H2O: 0.126, O2: 0.04, N2: 0.74\",\n \"C\": \"Dew Point: 110F, Mole Fractions: CO2: 0.1, H2O: 0.14, O2: 0.055, N2: 0.705\",\n \"D\": \"Dew Point: 115F, Mole Fractions: CO2: 0.105, H2O: 0.11, O2: 0.06, N2: 0.725\",\n \"E\": \"Dew Point: 142F, Mole Fractions: CO2: 0.088, H2O: 0.132, O2: 0.035, N2: 0.745\",\n \"F\": \"Dew Point: 125F, Mole Fractions: CO2: 0.092, H2O: 0.117, O2: 0.048, N2: 0.743\",\n \"G\": \"Dew Point: 120F, Mole Fractions: CO2: 0.1, H2O: 0.13, O2: 0.05, N2: 0.75\",\n \"H\": \"Dew Point: 145F, Mole Fractions: CO2: 0.09, H2O: 0.12, O2: 0.03, N2: 0.76\",\n \"I\": \"Dew Point: 150F, Mole Fractions: CO2: 0.085, H2O: 0.135, O2: 0.025, N2: 0.755\",\n \"J\": \"Dew Point: 130F, Mole Fractions: CO2: 0.095, H2O: 0.125, O2: 0.045, N2: 0.735\"\n }\n}
input
question.json
{
"question": "Propane burns in dry 125 percent theoretical air (21% O_2, 79% N_2) and the reaction is given by: C_3H_8 + 5O_2 + 18.8N_2 \\rightarrow 3CO_2 + 4H_2O + 18.8N_2 Calculate the dew point and the mole fractions of the combustionproducts when the air is at 1atmpressure.",
"options": {
"A": "Dew Point: 133F, Mole Fractions: CO2: 0.097, H2O: 0.123, O2: 0.042, N2: 0.738",
"B": "Dew Point: 138F, Mole Fractions: CO2: 0.094, H2O: 0.126, O2: 0.04, N2: 0.74",
"C": "Dew Point: 110F, Mole Fractions: CO2: 0.1, H2O: 0.14, O2: 0.055, N2: 0.705",
"D": "Dew Point: 115F, Mole Fractions: CO2: 0.105, H2O: 0.11, O2: 0.06, N2: 0.725",
"E": "Dew Point: 142F, Mole Fractions: CO2: 0.088, H2O: 0.132, O2: 0.035, N2: 0.745",
"F": "Dew Point: 125F, Mole Fractions: CO2: 0.092, H2O: 0.117, O2: 0.048, N2: 0.743",
"G": "Dew Point: 120F, Mole Fractions: CO2: 0.1, H2O: 0.13, O2: 0.05, N2: 0.75",
"H": "Dew Point: 145F, Mole Fractions: CO2: 0.09, H2O: 0.12, O2: 0.03, N2: 0.76",
"I": "Dew Point: 150F, Mole Fractions: CO2: 0.085, H2O: 0.135, O2: 0.025, N2: 0.755",
"J": "Dew Point: 130F, Mole Fractions: CO2: 0.095, H2O: 0.125, O2: 0.045, N2: 0.735"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_076/ files.
Scored by a judge this pinned commit does not carry.
▸case_077Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"The armature of a 230-volt shunt motor has a resistance of 0.18 ohm. If the armature current is not to exceed 76 amp, calculate: (a) the resistance that must be inserted in series with the armature at the instant of starting; (b) the value to which this resistance can be reduced when the armature accelerates until E_c is 168 volts; (c) the armature current at the instant of starting if no resistance is inserted in the armature circuit. (Assume a 2-volt drop at the brushes.)\",\n \"options\": {\n \"A\": \"(a) 3.05 ohms, (b) 0.65 ohm, (c) 1,278 amp\",\n \"B\": \"(a) 1.8 ohms, (b) 0.8 ohm, (c) 1,100 amp\",\n \"C\": \"(a) 3.1 ohms, (b) 0.75 ohm, (c) 1,500 amp\",\n \"D\": \"(a) 1.5 ohms, (b) 0.5 ohm, (c) 1,000 amp\",\n \"E\": \"(a) 2.3 ohms, (b) 0.58 ohm, (c) 1,225 amp\",\n \"F\": \"(a) 2.5 ohms, (b) 0.6 ohm, (c) 1,200 amp\",\n \"G\": \"(a) 2.9 ohms, (b) 0.55 ohm, (c) 1,350 amp\",\n \"H\": \"(a) 2.82 ohms, (b) 0.61 ohm, (c) 1,265 amp\",\n \"I\": \"(a) 2.0 ohms, (b) 0.7 ohm, (c) 1,150 amp\",\n \"J\": \"(a) 3.2 ohms, (b) 0.5 ohm, (c) 1,400 amp\"\n }\n}
input
question.json
{
"question": "The armature of a 230-volt shunt motor has a resistance of 0.18 ohm. If the armature current is not to exceed 76 amp, calculate: (a) the resistance that must be inserted in series with the armature at the instant of starting; (b) the value to which this resistance can be reduced when the armature accelerates until E_c is 168 volts; (c) the armature current at the instant of starting if no resistance is inserted in the armature circuit. (Assume a 2-volt drop at the brushes.)",
"options": {
"A": "(a) 3.05 ohms, (b) 0.65 ohm, (c) 1,278 amp",
"B": "(a) 1.8 ohms, (b) 0.8 ohm, (c) 1,100 amp",
"C": "(a) 3.1 ohms, (b) 0.75 ohm, (c) 1,500 amp",
"D": "(a) 1.5 ohms, (b) 0.5 ohm, (c) 1,000 amp",
"E": "(a) 2.3 ohms, (b) 0.58 ohm, (c) 1,225 amp",
"F": "(a) 2.5 ohms, (b) 0.6 ohm, (c) 1,200 amp",
"G": "(a) 2.9 ohms, (b) 0.55 ohm, (c) 1,350 amp",
"H": "(a) 2.82 ohms, (b) 0.61 ohm, (c) 1,265 amp",
"I": "(a) 2.0 ohms, (b) 0.7 ohm, (c) 1,150 amp",
"J": "(a) 3.2 ohms, (b) 0.5 ohm, (c) 1,400 amp"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_077/ files.
Scored by a judge this pinned commit does not carry.
▸case_078Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A 1(1/2) in sch. 40 steam pipe is laid in the atmosphere where the temperature is 50°F. The steam inside it is saturated at 100psia. Consider the pipe to be a grey body anduninsulated. The coefficient of heat transfer by natural convection from the outside surface is 2.0 Btu/hr sq ft R. Calculate the amount of steam con-densed per hour per unit length of pipe.\",\n \"options\": {\n \"A\": \"0.65 lb of steam/hr-ft of pipe\",\n \"B\": \"0.50 lb of steam/hr-ft of pipe\",\n \"C\": \"0.70 lb of steam/hr-ft of pipe\",\n \"D\": \"0.55 lb of steam/hr-ft of pipe\",\n \"E\": \"0.40 lb of steam/hr-ft of pipe\",\n \"F\": \"0.80 lb of steam/hr-ft of pipe\",\n \"G\": \"0.75 lb of steam/hr-ft of pipe\",\n \"H\": \"0.90 lb of steam/hr-ft of pipe\",\n \"I\": \"1.00 lb of steam/hr-ft of pipe\",\n \"J\": \"0.60 lb of steam/hr-ft of pipe\"\n }\n}
input
question.json
{
"question": "A 1(1/2) in sch. 40 steam pipe is laid in the atmosphere where the temperature is 50°F. The steam inside it is saturated at 100psia. Consider the pipe to be a grey body anduninsulated. The coefficient of heat transfer by natural convection from the outside surface is 2.0 Btu/hr sq ft R. Calculate the amount of steam con-densed per hour per unit length of pipe.",
"options": {
"A": "0.65 lb of steam/hr-ft of pipe",
"B": "0.50 lb of steam/hr-ft of pipe",
"C": "0.70 lb of steam/hr-ft of pipe",
"D": "0.55 lb of steam/hr-ft of pipe",
"E": "0.40 lb of steam/hr-ft of pipe",
"F": "0.80 lb of steam/hr-ft of pipe",
"G": "0.75 lb of steam/hr-ft of pipe",
"H": "0.90 lb of steam/hr-ft of pipe",
"I": "1.00 lb of steam/hr-ft of pipe",
"J": "0.60 lb of steam/hr-ft of pipe"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_078/ files.
Scored by a judge this pinned commit does not carry.
▸case_079Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A 72-slot three-phase stator armature is wound for six poles, usingdouble-layer lap coils having 20 turns per coil with a (5 / 6) pitch. The flux per pole is 4.8 × 10^6 lines, and the rotor speedis 1200 rpm. Calculate: (a) The generated effective voltage per coil of a full- pitch coil. (b) The total number of turns per phase. (c) The distribution factor. (d) The pitch factor. (e) The total generated voltage per phase from (a), (c), and (d)above, and by the following equation: E_gp= 4.44 \\\\textphiN_pfK_pk_d× 10^-8 V(1)\",\n \"options\": {\n \"A\": \"(a) 310 V/coil, (b) 520 turns/phase, (c) 0.945, (d) 0.960, (e) 5800 V\",\n \"B\": \"(a) 256 V/coil, (b) 480 turns/phase, (c) 0.958, (d) 0.966, (e) 5680 V\",\n \"C\": \"(a) 255 V/coil, (b) 475 turns/phase, (c) 0.955, (d) 0.965, (e) 5600 V\",\n \"D\": \"(a) 265 V/coil, (b) 500 turns/phase, (c) 0.955, (d) 0.975, (e) 5750 V\",\n \"E\": \"(a) 300 V/coil, (b) 500 turns/phase, (c) 0.950, (d) 0.970, (e) 5500 V\",\n \"F\": \"(a) 290 V/coil, (b) 470 turns/phase, (c) 0.960, (d) 0.980, (e) 5500 V\",\n \"G\": \"(a) 280 V/coil, (b) 460 turns/phase, (c) 0.950, (d) 0.970, (e) 5400 V\",\n \"H\": \"(a) 275 V/coil, (b) 485 turns/phase, (c) 0.945, (d) 0.955, (e) 5650 V\",\n \"I\": \"(a) 260 V/coil, (b) 490 turns/phase, (c) 0.960, (d) 0.965, (e) 5700 V\",\n \"J\": \"(a) 250 V/coil, (b) 510 turns/phase, (c) 0.962, (d) 0.968, (e) 5600 V\"\n }\n}
input
question.json
{
"question": "A 72-slot three-phase stator armature is wound for six poles, usingdouble-layer lap coils having 20 turns per coil with a (5 / 6) pitch. The flux per pole is 4.8 × 10^6 lines, and the rotor speedis 1200 rpm. Calculate: (a) The generated effective voltage per coil of a full- pitch coil. (b) The total number of turns per phase. (c) The distribution factor. (d) The pitch factor. (e) The total generated voltage per phase from (a), (c), and (d)above, and by the following equation: E_gp= 4.44 \\textphiN_pfK_pk_d× 10^-8 V(1)",
"options": {
"A": "(a) 310 V/coil, (b) 520 turns/phase, (c) 0.945, (d) 0.960, (e) 5800 V",
"B": "(a) 256 V/coil, (b) 480 turns/phase, (c) 0.958, (d) 0.966, (e) 5680 V",
"C": "(a) 255 V/coil, (b) 475 turns/phase, (c) 0.955, (d) 0.965, (e) 5600 V",
"D": "(a) 265 V/coil, (b) 500 turns/phase, (c) 0.955, (d) 0.975, (e) 5750 V",
"E": "(a) 300 V/coil, (b) 500 turns/phase, (c) 0.950, (d) 0.970, (e) 5500 V",
"F": "(a) 290 V/coil, (b) 470 turns/phase, (c) 0.960, (d) 0.980, (e) 5500 V",
"G": "(a) 280 V/coil, (b) 460 turns/phase, (c) 0.950, (d) 0.970, (e) 5400 V",
"H": "(a) 275 V/coil, (b) 485 turns/phase, (c) 0.945, (d) 0.955, (e) 5650 V",
"I": "(a) 260 V/coil, (b) 490 turns/phase, (c) 0.960, (d) 0.965, (e) 5700 V",
"J": "(a) 250 V/coil, (b) 510 turns/phase, (c) 0.962, (d) 0.968, (e) 5600 V"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_079/ files.
Scored by a judge this pinned commit does not carry.
▸case_080Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A witness lived next door to a victim. Late one night, the witness overheard the victim scream, \\\"The defendant, please don't do it!\\\" The next day the victim was found dead in her bedroom. The defendant was arrested and charged with murdering the victim. At trial, the witness proposes to testify to the victim's statement. Upon objection by defendant's counsel, the court should rule the witness's testimony regarding the victim's statement\",\n \"options\": {\n \"A\": \"admissible as a past recollection recorded if the witness can confirm the statement was made.\",\n \"B\": \"admissible as a dying declaration if the jury determines that the victim believed that her death was imminent.\",\n \"C\": \"admissible as a dying declaration if the judge, by preponderance of the evidence, determines that the victim believed that her death was imminent.\",\n \"D\": \"inadmissible, because the probative value is substantially outweighed by the danger of unfair prejudice.\",\n \"E\": \"inadmissible, because the statement was not made under oath.\",\n \"F\": \"inadmissible as hearsay not within any recognized exception.\",\n \"G\": \"inadmissible, because the statement is considered a character evidence.\",\n \"H\": \"admissible, because the statement is a present sense impression.\",\n \"I\": \"inadmissible, because the statement can lead to a character attack on the defendant.\"\n }\n}
input
question.json
{
"question": "A witness lived next door to a victim. Late one night, the witness overheard the victim scream, \"The defendant, please don't do it!\" The next day the victim was found dead in her bedroom. The defendant was arrested and charged with murdering the victim. At trial, the witness proposes to testify to the victim's statement. Upon objection by defendant's counsel, the court should rule the witness's testimony regarding the victim's statement",
"options": {
"A": "admissible as a past recollection recorded if the witness can confirm the statement was made.",
"B": "admissible as a dying declaration if the jury determines that the victim believed that her death was imminent.",
"C": "admissible as a dying declaration if the judge, by preponderance of the evidence, determines that the victim believed that her death was imminent.",
"D": "inadmissible, because the probative value is substantially outweighed by the danger of unfair prejudice.",
"E": "inadmissible, because the statement was not made under oath.",
"F": "inadmissible as hearsay not within any recognized exception.",
"G": "inadmissible, because the statement is considered a character evidence.",
"H": "admissible, because the statement is a present sense impression.",
"I": "inadmissible, because the statement can lead to a character attack on the defendant."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_080/ files.
Scored by a judge this pinned commit does not carry.
▸case_081Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A student at a private university sued the university in fed-eral court for negligence after he fell from scaffolding in a university-owned theater building. At trial, after briefing from both parties, the court permitted the jury to hear testimony that there had been several previous accidents in the same building. The jury found for the student, and the university appealed. One of the university's arguments on appeal is that the testimony about the previous accidents should have been excluded as irrelevant and highly prejudicial. Which standard of review applies to this argument?\",\n \"options\": {\n \"A\": \"Abuse of discretion.\",\n \"B\": \"Probable cause.\",\n \"C\": \"Harmless error.\",\n \"D\": \"Preponderance of the evidence.\",\n \"E\": \"Inevitable discovery.\",\n \"F\": \"Manifest injustice.\",\n \"G\": \"Fair and reasonable.\",\n \"H\": \"De novo.\",\n \"I\": \"Substantial evidence.\",\n \"J\": \"Clearly erroneous.\"\n }\n}
input
question.json
{
"question": "A student at a private university sued the university in fed-eral court for negligence after he fell from scaffolding in a university-owned theater building. At trial, after briefing from both parties, the court permitted the jury to hear testimony that there had been several previous accidents in the same building. The jury found for the student, and the university appealed. One of the university's arguments on appeal is that the testimony about the previous accidents should have been excluded as irrelevant and highly prejudicial. Which standard of review applies to this argument?",
"options": {
"A": "Abuse of discretion.",
"B": "Probable cause.",
"C": "Harmless error.",
"D": "Preponderance of the evidence.",
"E": "Inevitable discovery.",
"F": "Manifest injustice.",
"G": "Fair and reasonable.",
"H": "De novo.",
"I": "Substantial evidence.",
"J": "Clearly erroneous."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_081/ files.
Scored by a judge this pinned commit does not carry.
▸case_082Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A debtor owed a creditor $750 on an old debt. On July 1, the debt was barred by the statute of limitations. On August 1, the debtor ran into the creditor at a party and overheard him telling mutual friends that the debtor \\\"is a deadbeat who weiches on his debts. \\\" Feeling pangs of guilt, the debtor approached the creditor and orally agreed to pay him the $750 debt on September 1. The debtor refused to pay the creditor the $750 as promised on September 1St. If the creditor sues the debtor to recover the $750 debt, which would provide the strongest grounds that the debtor's oral promise was unenforceable?\",\n \"options\": {\n \"A\": \"There was no mutuality of obligation.\",\n \"B\": \"The creditor's claim was based on hearsay.\",\n \"C\": \"The promise was made out of guilt, not legal obligation.\",\n \"D\": \"It was violative of the statute of frauds.\",\n \"E\": \"The debtor was under duress when he made the promise.\",\n \"F\": \"The debtor had no legal capacity to enter into the agreement.\",\n \"G\": \"The debt was already barred by the statute of limitations.\",\n \"H\": \"The promise was not made in a formal legal setting.\",\n \"I\": \"It was not supported by new consideration.\",\n \"J\": \"The debtor did not provide a written agreement for the promise.\"\n }\n}
input
question.json
{
"question": "A debtor owed a creditor $750 on an old debt. On July 1, the debt was barred by the statute of limitations. On August 1, the debtor ran into the creditor at a party and overheard him telling mutual friends that the debtor \"is a deadbeat who weiches on his debts. \" Feeling pangs of guilt, the debtor approached the creditor and orally agreed to pay him the $750 debt on September 1. The debtor refused to pay the creditor the $750 as promised on September 1St. If the creditor sues the debtor to recover the $750 debt, which would provide the strongest grounds that the debtor's oral promise was unenforceable?",
"options": {
"A": "There was no mutuality of obligation.",
"B": "The creditor's claim was based on hearsay.",
"C": "The promise was made out of guilt, not legal obligation.",
"D": "It was violative of the statute of frauds.",
"E": "The debtor was under duress when he made the promise.",
"F": "The debtor had no legal capacity to enter into the agreement.",
"G": "The debt was already barred by the statute of limitations.",
"H": "The promise was not made in a formal legal setting.",
"I": "It was not supported by new consideration.",
"J": "The debtor did not provide a written agreement for the promise."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_082/ files.
Scored by a judge this pinned commit does not carry.
▸case_083Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"After a party where the adults all drank alcohol, with everyone having at least six beers, they started up a game of baseball. The host supplied the bats, balls and gloves. An argument arose at home plate after a controversial call by the umpire, who was inebriated like all of the other participants. The host of the party became enraged at the umpire's decision and hit him in the face, breaking his nose. The umpire pressed charges when he became sober, and the authorities charged the host with criminal battery. At trial, the defense raised the defense of voluntary intoxication, arguing that the host lacked the mentality to form an intent to commit a criminal battery. Will the defense of voluntary intoxication be successful under these facts?\",\n \"options\": {\n \"A\": \"Yes, voluntary intoxication is always at least a partial defense to any crime that requires mens rea.\",\n \"B\": \"No, criminal battery in this case is a general intent crime and the defense of voluntary intoxication is not available.\",\n \"C\": \"No, because the host was the organizer of the party and should have had control over the situation.\",\n \"D\": \"No, the host was grossly negligent in allowing a group of drunk men to play a dangerous game of hardball, which precluded him from the benefit of the defense.\",\n \"E\": \"Yes, criminal battery is a specific intent crime to which voluntary intoxication may be asserted as a defense.\",\n \"F\": \"No, because the host willingly drank the alcohol and should have known the possible consequences.\",\n \"G\": \"Yes, because the umpire was also intoxicated and could have misjudged the situation.\",\n \"H\": \"No, because the host used excessive force in a situation that did not warrant it.\",\n \"I\": \"Yes, because the umpire provoked the host by making an unfair call.\",\n \"J\": \"Yes, because the host was not in his right mind due to the alcohol.\"\n }\n}
input
question.json
{
"question": "After a party where the adults all drank alcohol, with everyone having at least six beers, they started up a game of baseball. The host supplied the bats, balls and gloves. An argument arose at home plate after a controversial call by the umpire, who was inebriated like all of the other participants. The host of the party became enraged at the umpire's decision and hit him in the face, breaking his nose. The umpire pressed charges when he became sober, and the authorities charged the host with criminal battery. At trial, the defense raised the defense of voluntary intoxication, arguing that the host lacked the mentality to form an intent to commit a criminal battery. Will the defense of voluntary intoxication be successful under these facts?",
"options": {
"A": "Yes, voluntary intoxication is always at least a partial defense to any crime that requires mens rea.",
"B": "No, criminal battery in this case is a general intent crime and the defense of voluntary intoxication is not available.",
"C": "No, because the host was the organizer of the party and should have had control over the situation.",
"D": "No, the host was grossly negligent in allowing a group of drunk men to play a dangerous game of hardball, which precluded him from the benefit of the defense.",
"E": "Yes, criminal battery is a specific intent crime to which voluntary intoxication may be asserted as a defense.",
"F": "No, because the host willingly drank the alcohol and should have known the possible consequences.",
"G": "Yes, because the umpire was also intoxicated and could have misjudged the situation.",
"H": "No, because the host used excessive force in a situation that did not warrant it.",
"I": "Yes, because the umpire provoked the host by making an unfair call.",
"J": "Yes, because the host was not in his right mind due to the alcohol."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_083/ files.
Scored by a judge this pinned commit does not carry.
▸case_084Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A football player was the star fulllack for the local college football team. After missing two practices, the football player was dropped from the team by the head football coach. Following his dismissal, the football player met with the coach and asked if he could rejoin the team. The coach said that the football player was despised by the other players and under no circumstances could he return to the team. As the football player was leaving the coach's office, feeling very dejected, the coach then said to him, \\\"Hope you decide to transfer, because everybody hates your guts around here. \\\"Later that same evening, the football player wrote a suicide note in which he stated, \\\"The coach is responsible for my despondency. If I can't play football, I don't want to live. \\\" After swallowing a bottle of Quaalude barbiturates, the football player fell unconscious in his dormitory room. Moments later, the football player's roommate entered the room and saw his limp body on the floor. The roommate read the suicide note and then attempted to administer aid. Failing to revive him, the roommate picked up the football player and carried him to the college's first aid center. The football player received prompt medical attention and soon recovered from his drug overdose. If the football player asserts a claim against the coach based on intentional infliction of emotional distress, the football player will most likely\",\n \"options\": {\n \"A\": \"not prevail, because the coach had the right to express his opinion.\",\n \"B\": \"prevail, because the coach's remarks led directly to the football player's emotional distress and subsequent suicide attempt.\",\n \"C\": \"prevail, because the coach intended to cause him to suffer emotional distress.\",\n \"D\": \"not prevail, because the football player's drug overdose resulted from his own voluntary act.\",\n \"E\": \"prevail, because the coach's remarks constituted bullying behavior.\",\n \"F\": \"prevail, because the coach's remarks were intended to inflict emotional distress and resulted in the football player's suicide attempt.\",\n \"G\": \"not prevail, because the coach acted reasonably under the circumstances, since everyone on the team hated the football player.\",\n \"H\": \"prevail, because the coach's remark did, in fact, cause the football player to suffer emotional distress.\",\n \"I\": \"not prevail, because the coach's remarks do not meet the legal standard for intentional infliction of emotional distress.\",\n \"J\": \"not prevail, because the football player's overdose was an unforeseen consequence of the coach's remarks.\"\n }\n}
input
question.json
{
"question": "A football player was the star fulllack for the local college football team. After missing two practices, the football player was dropped from the team by the head football coach. Following his dismissal, the football player met with the coach and asked if he could rejoin the team. The coach said that the football player was despised by the other players and under no circumstances could he return to the team. As the football player was leaving the coach's office, feeling very dejected, the coach then said to him, \"Hope you decide to transfer, because everybody hates your guts around here. \"Later that same evening, the football player wrote a suicide note in which he stated, \"The coach is responsible for my despondency. If I can't play football, I don't want to live. \" After swallowing a bottle of Quaalude barbiturates, the football player fell unconscious in his dormitory room. Moments later, the football player's roommate entered the room and saw his limp body on the floor. The roommate read the suicide note and then attempted to administer aid. Failing to revive him, the roommate picked up the football player and carried him to the college's first aid center. The football player received prompt medical attention and soon recovered from his drug overdose. If the football player asserts a claim against the coach based on intentional infliction of emotional distress, the football player will most likely",
"options": {
"A": "not prevail, because the coach had the right to express his opinion.",
"B": "prevail, because the coach's remarks led directly to the football player's emotional distress and subsequent suicide attempt.",
"C": "prevail, because the coach intended to cause him to suffer emotional distress.",
"D": "not prevail, because the football player's drug overdose resulted from his own voluntary act.",
"E": "prevail, because the coach's remarks constituted bullying behavior.",
"F": "prevail, because the coach's remarks were intended to inflict emotional distress and resulted in the football player's suicide attempt.",
"G": "not prevail, because the coach acted reasonably under the circumstances, since everyone on the team hated the football player.",
"H": "prevail, because the coach's remark did, in fact, cause the football player to suffer emotional distress.",
"I": "not prevail, because the coach's remarks do not meet the legal standard for intentional infliction of emotional distress.",
"J": "not prevail, because the football player's overdose was an unforeseen consequence of the coach's remarks."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_084/ files.
Scored by a judge this pinned commit does not carry.
▸case_085Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A widow owned an apartment building. Several years ago, a student, in a signed writing, entered a three-year lease agreement with the widow. The student agreed to pay $800 monthly rent for the third-floor apartment in the widow's building. The original three-year term had long since expired, but the student had continued to pay his rent, and the widow continued to accept it. If the widow had chosen to evict the student immediately upon the end of the three-year period, and before she had accepted the next rent check, the student's strongest argument to prevent eviction, if true, would have been that\",\n \"options\": {\n \"A\": \"the student had a right to stay as he had been residing there for more than three years.\",\n \"B\": \"the widow had not given the student requisitenotice for terminating a term of years.\",\n \"C\": \"the student had made improvements to the property, thus extending the leasehold.\",\n \"D\": \"the student had a verbal agreement with the widow to extend the lease.\",\n \"E\": \"the terms of the leasehold contract were binding on both parties.\",\n \"F\": \"the widow had accepted rent for the following month before the three-year period ended.\",\n \"G\": \"the student was protected from eviction due to his status as a student.\",\n \"H\": \"the widow had not clearly communicated the end of the lease term.\",\n \"I\": \"the widow had not given the student requisitenotice for terminating a periodic tenancy.\",\n \"J\": \"the leasehold contract was void ab initio.\"\n }\n}
input
question.json
{
"question": "A widow owned an apartment building. Several years ago, a student, in a signed writing, entered a three-year lease agreement with the widow. The student agreed to pay $800 monthly rent for the third-floor apartment in the widow's building. The original three-year term had long since expired, but the student had continued to pay his rent, and the widow continued to accept it. If the widow had chosen to evict the student immediately upon the end of the three-year period, and before she had accepted the next rent check, the student's strongest argument to prevent eviction, if true, would have been that",
"options": {
"A": "the student had a right to stay as he had been residing there for more than three years.",
"B": "the widow had not given the student requisitenotice for terminating a term of years.",
"C": "the student had made improvements to the property, thus extending the leasehold.",
"D": "the student had a verbal agreement with the widow to extend the lease.",
"E": "the terms of the leasehold contract were binding on both parties.",
"F": "the widow had accepted rent for the following month before the three-year period ended.",
"G": "the student was protected from eviction due to his status as a student.",
"H": "the widow had not clearly communicated the end of the lease term.",
"I": "the widow had not given the student requisitenotice for terminating a periodic tenancy.",
"J": "the leasehold contract was void ab initio."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_085/ files.
Scored by a judge this pinned commit does not carry.
▸case_086Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Atoroidaltype of magnetic core of rectangular cross section hasthe following parameter values: l_m= 20 cm, A_m = (2 cm) (1 cm) = 2 cm^2, l_g= 0.1 cm,\\\\mu_r= 4000. Determine the ampere turnsiNrequired to realize a magneticflux density in the air gap of magnitude B_g= 0.1Wb/m^2. The air-gap area is taken as A_g = (2 + 0.1) (1 + 0.1) cm^2 = 2.31 cm^2.\",\n \"options\": {\n \"A\": \"110 ampere turns\",\n \"B\": \"90 ampere turns\",\n \"C\": \"150 ampere turns\",\n \"D\": \"120 ampere turns\",\n \"E\": \"100 ampere turns\",\n \"F\": \"75 ampere turns\",\n \"G\": \"95.5 ampere turns\",\n \"H\": \"70 ampere turns\",\n \"I\": \"84.1 ampere turns\",\n \"J\": \"65 ampere turns\"\n }\n}
input
question.json
{
"question": "Atoroidaltype of magnetic core of rectangular cross section hasthe following parameter values: l_m= 20 cm, A_m = (2 cm) (1 cm) = 2 cm^2, l_g= 0.1 cm,\\mu_r= 4000. Determine the ampere turnsiNrequired to realize a magneticflux density in the air gap of magnitude B_g= 0.1Wb/m^2. The air-gap area is taken as A_g = (2 + 0.1) (1 + 0.1) cm^2 = 2.31 cm^2.",
"options": {
"A": "110 ampere turns",
"B": "90 ampere turns",
"C": "150 ampere turns",
"D": "120 ampere turns",
"E": "100 ampere turns",
"F": "75 ampere turns",
"G": "95.5 ampere turns",
"H": "70 ampere turns",
"I": "84.1 ampere turns",
"J": "65 ampere turns"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_086/ files.
Scored by a judge this pinned commit does not carry.
▸case_087Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"What is the maximum air velocity that will not exceed a 2°F temperature rise at a stagnation point ? Assume air at 70°F and 14.7psia. TABLE Gas Chemical Formula Specific Weight, \\\\gamma Density, \\\\rho Gas Constant, R Adiabatic Constant, k lb/ft^3 N/m^3 slugs/ft^3 kg/m^3 ft/°R m/°K Air --- 0.0753 11.8 0.00234 1.206 53.3 29.2 1.40 Ammonia NH_3 0.0448 7.04 0.00139 0.716 89.5 49.1 1.32 Carbon Dioxide CO_2 0.115 18.1 0.00357 1.840 34.9 19.1 1.29 Helium He 0.0104 1.63 0.000323 0.166 386. 212. 1.66 Hydrogen H_2 0.00522 0.820 0.000162 0.0835 767. 421. 1.40 Methane CH_4 0.0416 6.53 0.00129 0.665 96.4 52.9 1.32 Nitrogen N_2 0.0726 11.4 0.00225 1.160 55.2 30.3 1.40 Oxygen O_2 0.0830 13.0 0.00258 1.330 48.3 26.5 1.40 Sulfur dioxide SO_2 0.170 26.7 0.00528 2.721 23.6 12.9 1.26\",\n \"options\": {\n \"A\": \"225 ft/s\",\n \"B\": \"250 ft/s\",\n \"C\": \"100 ft/s\",\n \"D\": \"200 ft/s\",\n \"E\": \"153 ft/s\",\n \"F\": \"300 ft/s\",\n \"G\": \"50 ft/s\",\n \"H\": \"180 ft/s\",\n \"I\": \"125 ft/s\",\n \"J\": \"75 ft/s\"\n }\n}
input
question.json
{
"question": "What is the maximum air velocity that will not exceed a 2°F temperature rise at a stagnation point ? Assume air at 70°F and 14.7psia. TABLE Gas Chemical Formula Specific Weight, \\gamma Density, \\rho Gas Constant, R Adiabatic Constant, k lb/ft^3 N/m^3 slugs/ft^3 kg/m^3 ft/°R m/°K Air --- 0.0753 11.8 0.00234 1.206 53.3 29.2 1.40 Ammonia NH_3 0.0448 7.04 0.00139 0.716 89.5 49.1 1.32 Carbon Dioxide CO_2 0.115 18.1 0.00357 1.840 34.9 19.1 1.29 Helium He 0.0104 1.63 0.000323 0.166 386. 212. 1.66 Hydrogen H_2 0.00522 0.820 0.000162 0.0835 767. 421. 1.40 Methane CH_4 0.0416 6.53 0.00129 0.665 96.4 52.9 1.32 Nitrogen N_2 0.0726 11.4 0.00225 1.160 55.2 30.3 1.40 Oxygen O_2 0.0830 13.0 0.00258 1.330 48.3 26.5 1.40 Sulfur dioxide SO_2 0.170 26.7 0.00528 2.721 23.6 12.9 1.26",
"options": {
"A": "225 ft/s",
"B": "250 ft/s",
"C": "100 ft/s",
"D": "200 ft/s",
"E": "153 ft/s",
"F": "300 ft/s",
"G": "50 ft/s",
"H": "180 ft/s",
"I": "125 ft/s",
"J": "75 ft/s"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_087/ files.
Scored by a judge this pinned commit does not carry.
▸case_088Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A pharmacist was employed by a drug store. A long-time customer of the drug store presented a prescription to the pharmacist for the medication Clinoril, which was prescribed by her physician to treat her arthritic condition. The pharmacist was in a rush that day because of a backlog of prescriptions to fill, and he misread the customer's prescription. He filled her prescription with Clinitest, a caustic substance not for internal use. He labeled the container with the instructions for Clinoril: \\\"Take one tablet twice a day. \\\"The customer followed the instructions, consumed the Clinitest, and suffered severe injuries to her stomach and esophagus. If the customer brings a strict liability in tort action against the pharmacist, what is his best defense?\",\n \"options\": {\n \"A\": \"The customer did not suffer any injuries due to the medication.\",\n \"B\": \"The customer did not ask for clarification about the medication.\",\n \"C\": \"The physician had prescribed the wrong medication.\",\n \"D\": \"The drug store was not the manufacturer upon whom ultimate liability falls.\",\n \"E\": \"The medication was properly labeled with the correct medication name.\",\n \"F\": \"It was the cashier and not he who personally received the money for the medication.\",\n \"G\": \"The customer should have been aware that the medication was not correct.\",\n \"H\": \"He was not a seller of the product upon whom strict liability may be imposed.\",\n \"I\": \"He exercised reasonable care under the circumstances.\",\n \"J\": \"The customer misread the label on the medication.\"\n }\n}
input
question.json
{
"question": "A pharmacist was employed by a drug store. A long-time customer of the drug store presented a prescription to the pharmacist for the medication Clinoril, which was prescribed by her physician to treat her arthritic condition. The pharmacist was in a rush that day because of a backlog of prescriptions to fill, and he misread the customer's prescription. He filled her prescription with Clinitest, a caustic substance not for internal use. He labeled the container with the instructions for Clinoril: \"Take one tablet twice a day. \"The customer followed the instructions, consumed the Clinitest, and suffered severe injuries to her stomach and esophagus. If the customer brings a strict liability in tort action against the pharmacist, what is his best defense?",
"options": {
"A": "The customer did not suffer any injuries due to the medication.",
"B": "The customer did not ask for clarification about the medication.",
"C": "The physician had prescribed the wrong medication.",
"D": "The drug store was not the manufacturer upon whom ultimate liability falls.",
"E": "The medication was properly labeled with the correct medication name.",
"F": "It was the cashier and not he who personally received the money for the medication.",
"G": "The customer should have been aware that the medication was not correct.",
"H": "He was not a seller of the product upon whom strict liability may be imposed.",
"I": "He exercised reasonable care under the circumstances.",
"J": "The customer misread the label on the medication."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_088/ files.
Scored by a judge this pinned commit does not carry.
▸case_089Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A father lived with his son, who was addicted to crack cocaine. Under its influence, the son often becameviolent and physically abused his father. As a result, the father always lived in fear. One night, the fatherheard his son on the front stoop making loud obscene remarks. The father was certain that his son wasunder the influence of crack and was terrified that he would be physically beaten again. In his fear, hebolted the front door and took out a revolver. When the son discovered that the door was bolted, he kickedit down. As the son burst through the front door, his father shot him four times in the chest, killing him. Infact, the son was not under the influence of crack or any drug and did not intend to harm his father. At trial, the father presented the above facts and asked the judge to instruct the jury on self-defense. How should the judge instruct the jury with respect to self-defense?\",\n \"options\": {\n \"A\": \"Give the self-defense instruction, because the son forcibly entered the home.\",\n \"B\": \"Deny the self-defense instruction, because the son was not actually under the influence of drugs.\",\n \"C\": \"Give the self-defense instruction, because the father had a reasonable fear for his life.\",\n \"D\": \"Give the self-defense instruction, because it expresses the defense's theory of the case.\",\n \"E\": \"Give the self-defense instruction, because the father had previously been victimized by his son.\",\n \"F\": \"Deny the self-defense instruction, because the father had other options to protect himself.\",\n \"G\": \"Give the self-defense instruction, because the evidence was sufficient to raise the defense.\",\n \"H\": \"Deny the self-defense instruction, because the son was unarmed.\",\n \"I\": \"Deny the self-defense instruction, because the father used excessive force.\",\n \"J\": \"Deny the self-defense instruction, because the father was not in imminent danger from his son.\"\n }\n}
input
question.json
{
"question": "A father lived with his son, who was addicted to crack cocaine. Under its influence, the son often becameviolent and physically abused his father. As a result, the father always lived in fear. One night, the fatherheard his son on the front stoop making loud obscene remarks. The father was certain that his son wasunder the influence of crack and was terrified that he would be physically beaten again. In his fear, hebolted the front door and took out a revolver. When the son discovered that the door was bolted, he kickedit down. As the son burst through the front door, his father shot him four times in the chest, killing him. Infact, the son was not under the influence of crack or any drug and did not intend to harm his father. At trial, the father presented the above facts and asked the judge to instruct the jury on self-defense. How should the judge instruct the jury with respect to self-defense?",
"options": {
"A": "Give the self-defense instruction, because the son forcibly entered the home.",
"B": "Deny the self-defense instruction, because the son was not actually under the influence of drugs.",
"C": "Give the self-defense instruction, because the father had a reasonable fear for his life.",
"D": "Give the self-defense instruction, because it expresses the defense's theory of the case.",
"E": "Give the self-defense instruction, because the father had previously been victimized by his son.",
"F": "Deny the self-defense instruction, because the father had other options to protect himself.",
"G": "Give the self-defense instruction, because the evidence was sufficient to raise the defense.",
"H": "Deny the self-defense instruction, because the son was unarmed.",
"I": "Deny the self-defense instruction, because the father used excessive force.",
"J": "Deny the self-defense instruction, because the father was not in imminent danger from his son."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_089/ files.
Scored by a judge this pinned commit does not carry.
▸case_090Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A landlord, an owner in fee simple, leased a three- story house to a tenant for a term of 10 years. By the terms of the lease, the tenant expressly covenanted to pay a monthly rental of $300 and to pay the taxes on the premises during the term of the lease. The lease also stipulated that the tenant, as lessee, may not assign or sublease the said premises. The tenant and his family lived in the house for two years. Then the tenant, still owing the landlord three months back rent, assigned his leasehold interest in the property to a student, who was unaware of the prohibition against assignments. This written assignment expressly provided that the student would pay the landlord the monthly rental of $300, but was silent concerning the payment of taxes. The landlord never objected to this assignment. Six months after the assignment, the landlord conveyed his property interest in the premises to a new owner. The student failed to pay the taxes on the property. The new owner, after paying the taxes, brought suit against the student for the amount paid. Judgment should be for\",\n \"options\": {\n \"A\": \"the student, because the new owner should have notified him about the taxes.\",\n \"B\": \"the new owner.\",\n \"C\": \"the landlord, because he never objected to the assignment.\",\n \"D\": \"the student, because he was not aware of the original lease agreement.\",\n \"E\": \"the new owner, because the student was not part of the original lease agreement.\",\n \"F\": \"the student, because the covenant to pay taxes will not \\\"run with the land,\\\" unless the intent is clearly expressed in the original lease.\",\n \"G\": \"the tenant, because he was the one who originally agreed to pay the taxes.\",\n \"H\": \"the student, because he did not contract with the tenant to pay the taxes on the property.\",\n \"I\": \"the student, because the covenant to pay taxes is merely collateral and does not \\\"run with the land. \\\"\"\n }\n}
input
question.json
{
"question": "A landlord, an owner in fee simple, leased a three- story house to a tenant for a term of 10 years. By the terms of the lease, the tenant expressly covenanted to pay a monthly rental of $300 and to pay the taxes on the premises during the term of the lease. The lease also stipulated that the tenant, as lessee, may not assign or sublease the said premises. The tenant and his family lived in the house for two years. Then the tenant, still owing the landlord three months back rent, assigned his leasehold interest in the property to a student, who was unaware of the prohibition against assignments. This written assignment expressly provided that the student would pay the landlord the monthly rental of $300, but was silent concerning the payment of taxes. The landlord never objected to this assignment. Six months after the assignment, the landlord conveyed his property interest in the premises to a new owner. The student failed to pay the taxes on the property. The new owner, after paying the taxes, brought suit against the student for the amount paid. Judgment should be for",
"options": {
"A": "the student, because the new owner should have notified him about the taxes.",
"B": "the new owner.",
"C": "the landlord, because he never objected to the assignment.",
"D": "the student, because he was not aware of the original lease agreement.",
"E": "the new owner, because the student was not part of the original lease agreement.",
"F": "the student, because the covenant to pay taxes will not \"run with the land,\" unless the intent is clearly expressed in the original lease.",
"G": "the tenant, because he was the one who originally agreed to pay the taxes.",
"H": "the student, because he did not contract with the tenant to pay the taxes on the property.",
"I": "the student, because the covenant to pay taxes is merely collateral and does not \"run with the land. \""
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_090/ files.
Scored by a judge this pinned commit does not carry.
▸case_091Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A drop of water is allowed to fall into dry still air at 1atmpressure with a velocity of 225 cm sec^-1.The drop is sphericalin shape with 0.03cm diameter. The temperature of airis 145°F and the surface temperature of the drop is 70°F. The vapor pressure of water at 70°F is 0.0247 atm. Assuming pseudo steady state process, calculate the net rateof evaporation from the water drop.\",\n \"options\": {\n \"A\": \"1.324 × 10^-8gmole/sec\",\n \"B\": \"2.5 × 10^-7 gmole/sec\",\n \"C\": \"6.62 × 10^-7 gmole/sec\",\n \"D\": \"2.648 × 10^-6 gmole/sec\",\n \"E\": \"1.324 × 10^-6gmole/sec\",\n \"F\": \"1.324 × 10^-7gmole/sec\",\n \"G\": \"6.62 × 10^-8 gmole/sec\",\n \"H\": \"1.5 × 10^-7gmole/sec\",\n \"I\": \"0.75 × 10^-7 gmole/sec\",\n \"J\": \"2.124 × 10^-8 gmole/sec\"\n }\n}
input
question.json
{
"question": "A drop of water is allowed to fall into dry still air at 1atmpressure with a velocity of 225 cm sec^-1.The drop is sphericalin shape with 0.03cm diameter. The temperature of airis 145°F and the surface temperature of the drop is 70°F. The vapor pressure of water at 70°F is 0.0247 atm. Assuming pseudo steady state process, calculate the net rateof evaporation from the water drop.",
"options": {
"A": "1.324 × 10^-8gmole/sec",
"B": "2.5 × 10^-7 gmole/sec",
"C": "6.62 × 10^-7 gmole/sec",
"D": "2.648 × 10^-6 gmole/sec",
"E": "1.324 × 10^-6gmole/sec",
"F": "1.324 × 10^-7gmole/sec",
"G": "6.62 × 10^-8 gmole/sec",
"H": "1.5 × 10^-7gmole/sec",
"I": "0.75 × 10^-7 gmole/sec",
"J": "2.124 × 10^-8 gmole/sec"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_091/ files.
Scored by a judge this pinned commit does not carry.
▸case_092Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"The directivity of an antenna is 50 and the antenna operates ata wavelength of 4 meters. What is its maximumeffective aperture?\",\n \"options\": {\n \"A\": \"100 meter^2\",\n \"B\": \"50 meter^2\",\n \"C\": \"12.7 meter^2\",\n \"D\": \"200 meter^2\",\n \"E\": \"25 meter^2\",\n \"F\": \"4 meter^2\",\n \"G\": \"80 meter^2\",\n \"H\": \"63.6 meter^2\",\n \"I\": \"16 meter^2\",\n \"J\": \"31.8 meter^2\"\n }\n}
input
question.json
{
"question": "The directivity of an antenna is 50 and the antenna operates ata wavelength of 4 meters. What is its maximumeffective aperture?",
"options": {
"A": "100 meter^2",
"B": "50 meter^2",
"C": "12.7 meter^2",
"D": "200 meter^2",
"E": "25 meter^2",
"F": "4 meter^2",
"G": "80 meter^2",
"H": "63.6 meter^2",
"I": "16 meter^2",
"J": "31.8 meter^2"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_092/ files.
Scored by a judge this pinned commit does not carry.
▸case_093Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"After his 16th birthday, a youth decided to purchase a new sports car from a dealership for the sales price of $36,000. The youth thereafter entered into a written contract with the dealership, which provided that the youth would make monthly payments of $1,000 for three years. During the first year of the contract, the youth made 12 monthly payments totaling $12,000. However, the youth failed to make any payments during the second year of the contract. The dealership repeatedly threatened to sue the youth for the remaining balance that it claimed was due under the contract. Finally, in order to avoid litigation, the youth sent the dealership the following letter:\\\"The car I purchased from you is a real lemon. I have spent a considerable sum of money during the last two years in repair work. I don't believe that the car is worth $36,000, but I am willing to pay you $10,000 if I find ajob in the next month. \\\"The youth had just turned 18 when he sent this letter to the dealership. The day after mailing this letter, the youth was involved in an automobile accident, and the sports car was totally demolished. Following the accident, the youth remained unemployed, and he refused to make any payments to the dealership. At the time of the accident, the sports car was worth $18,000. In this jurisdiction, the age of majority is18 years of age. If the dealership brings suit against the youth for breach of contract, what, if any, is the plaintiff's proper recovery?\",\n \"options\": {\n \"A\": \"$36,000.00\",\n \"B\": \"$10,000. 00\",\n \"C\": \"$18,000. 00\",\n \"D\": \"The cost of repair work the youth has done on the car.\",\n \"E\": \"$30,000.00\",\n \"F\": \"Nothing.\",\n \"G\": \"$12,000.00\",\n \"H\": \"The cost of the car at the time of purchase minus the payments made so far.\",\n \"I\": \"$6,000.00\"\n }\n}
input
question.json
{
"question": "After his 16th birthday, a youth decided to purchase a new sports car from a dealership for the sales price of $36,000. The youth thereafter entered into a written contract with the dealership, which provided that the youth would make monthly payments of $1,000 for three years. During the first year of the contract, the youth made 12 monthly payments totaling $12,000. However, the youth failed to make any payments during the second year of the contract. The dealership repeatedly threatened to sue the youth for the remaining balance that it claimed was due under the contract. Finally, in order to avoid litigation, the youth sent the dealership the following letter:\"The car I purchased from you is a real lemon. I have spent a considerable sum of money during the last two years in repair work. I don't believe that the car is worth $36,000, but I am willing to pay you $10,000 if I find ajob in the next month. \"The youth had just turned 18 when he sent this letter to the dealership. The day after mailing this letter, the youth was involved in an automobile accident, and the sports car was totally demolished. Following the accident, the youth remained unemployed, and he refused to make any payments to the dealership. At the time of the accident, the sports car was worth $18,000. In this jurisdiction, the age of majority is18 years of age. If the dealership brings suit against the youth for breach of contract, what, if any, is the plaintiff's proper recovery?",
"options": {
"A": "$36,000.00",
"B": "$10,000. 00",
"C": "$18,000. 00",
"D": "The cost of repair work the youth has done on the car.",
"E": "$30,000.00",
"F": "Nothing.",
"G": "$12,000.00",
"H": "The cost of the car at the time of purchase minus the payments made so far.",
"I": "$6,000.00"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_093/ files.
Scored by a judge this pinned commit does not carry.
▸case_094Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A state enacts a statute that prohibits \\\"anyone over60 years of age to run for public office. \\\" A state senator has been in office for three terms and wishes to seek re-election. The senator, who is 61, brings suit challenging the constitutionality of the state statute. Which of the following best states the burden of persuasion?\",\n \"options\": {\n \"A\": \"The petitioner must show the statute violates due process by depriving her of the right to be a candidate.\",\n \"B\": \"The state needs to show that the statute is rationally related to a compelling government interest.\",\n \"C\": \"Since a fundamental right is involved, the petitioner must show the regulation is necessary to vindicate a compelling government interest.\",\n \"D\": \"Since no fundamental right is involved, the state must show the age restriction is essential to maintain a legitimate government interest.\",\n \"E\": \"The state must prove that the age restriction is essential to maintain the efficiency of the government.\",\n \"F\": \"The petitioner must show the statute violates the fundamental right of freedom of expression.\",\n \"G\": \"Since no fundamental right is involved, the petitioner must show the age restriction is not rationally related to a legitimate government interest.\"\n }\n}
input
question.json
{
"question": "A state enacts a statute that prohibits \"anyone over60 years of age to run for public office. \" A state senator has been in office for three terms and wishes to seek re-election. The senator, who is 61, brings suit challenging the constitutionality of the state statute. Which of the following best states the burden of persuasion?",
"options": {
"A": "The petitioner must show the statute violates due process by depriving her of the right to be a candidate.",
"B": "The state needs to show that the statute is rationally related to a compelling government interest.",
"C": "Since a fundamental right is involved, the petitioner must show the regulation is necessary to vindicate a compelling government interest.",
"D": "Since no fundamental right is involved, the state must show the age restriction is essential to maintain a legitimate government interest.",
"E": "The state must prove that the age restriction is essential to maintain the efficiency of the government.",
"F": "The petitioner must show the statute violates the fundamental right of freedom of expression.",
"G": "Since no fundamental right is involved, the petitioner must show the age restriction is not rationally related to a legitimate government interest."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_094/ files.
Scored by a judge this pinned commit does not carry.
▸case_095Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A box which tells the effect of inputs on control sub system is known as\",\n \"options\": {\n \"A\": \"Process box.\",\n \"B\": \"State box.\",\n \"C\": \"Input box.\",\n \"D\": \"System box.\",\n \"E\": \"Data Box.\",\n \"F\": \"Control box.\",\n \"G\": \"Output box.\",\n \"H\": \"Decision box.\",\n \"I\": \"Effect box.\",\n \"J\": \"Logical box.\"\n }\n}
input
question.json
{
"question": "A box which tells the effect of inputs on control sub system is known as",
"options": {
"A": "Process box.",
"B": "State box.",
"C": "Input box.",
"D": "System box.",
"E": "Data Box.",
"F": "Control box.",
"G": "Output box.",
"H": "Decision box.",
"I": "Effect box.",
"J": "Logical box."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_095/ files.
Scored by a judge this pinned commit does not carry.
▸case_096Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A preacher and his posse make a habit of showing up at funerals of soldiers suspected to be gay, carrying obnoxious signs and yelling out hurtful epithets. Finally, the parents of one dead soldier has enough and sues the preacher, and the case goes all the way to the Supreme Court. Would the Court likely find in favor of the preacher or the father?\",\n \"options\": {\n \"A\": \"The father, because the preacher's actions disturbed the peace of a private event.\",\n \"B\": \"The preacher, because it involves a matter of public interest.\",\n \"C\": \"The preacher, because the First Amendment protects even offensive or hateful speech.\",\n \"D\": \"The father, as a funeral is a private affair.\",\n \"E\": \"The preacher, because his actions are protected under freedom of speech and religion.\",\n \"F\": \"The father, because the preacher's actions were intentionally disruptive and disrespectful.\",\n \"G\": \"The preacher, because he has a right to express his religious views regardless of the context.\",\n \"H\": \"The father, because the First Amendment does not include protection for vicious verbal assaults such as this.\",\n \"I\": \"The father, because the preacher's actions constitute harassment and are therefore not protected by the First Amendment.\"\n }\n}
input
question.json
{
"question": "A preacher and his posse make a habit of showing up at funerals of soldiers suspected to be gay, carrying obnoxious signs and yelling out hurtful epithets. Finally, the parents of one dead soldier has enough and sues the preacher, and the case goes all the way to the Supreme Court. Would the Court likely find in favor of the preacher or the father?",
"options": {
"A": "The father, because the preacher's actions disturbed the peace of a private event.",
"B": "The preacher, because it involves a matter of public interest.",
"C": "The preacher, because the First Amendment protects even offensive or hateful speech.",
"D": "The father, as a funeral is a private affair.",
"E": "The preacher, because his actions are protected under freedom of speech and religion.",
"F": "The father, because the preacher's actions were intentionally disruptive and disrespectful.",
"G": "The preacher, because he has a right to express his religious views regardless of the context.",
"H": "The father, because the First Amendment does not include protection for vicious verbal assaults such as this.",
"I": "The father, because the preacher's actions constitute harassment and are therefore not protected by the First Amendment."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_096/ files.
Scored by a judge this pinned commit does not carry.
▸case_097Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A condominium development consists of two buildings, one with balconies attached to each unit, and one withno balconies. For safety concerns, the condominium association amended the covenants and restrictions toprohibit future sales of balcony units to families with minor children. The amendment did not affect families withchildren already living in balcony units. The amendment was promptly recorded. The condominium associationhad a valid covenant providing that all sales had to be approved by the association. Subsequent to the effective date of the amendment, the owner of a balcony unit contracted to sell it to a familywith minor children. Before the closing, the association told the buyers that because they had minor children,they could not buy the unit. The association further told the buyers that numerous units were available in thebuilding without balconies. After receiving this notification, the buyers complained to a fair housing agency, claiming that the amendmentwas unenforceable because it violated federal fair housing laws. Is there reasonable cause to believe that a violation has occurred?\",\n \"options\": {\n \"A\": \"Yes, because families with children are already living in units with balconies.\",\n \"B\": \"No, because the amendment was promptly recorded and is legally binding.\",\n \"C\": \"Yes, because families with children cannot be segregated within the condominium development.\",\n \"D\": \"No, because the association is acting in the best interests of child safety.\",\n \"E\": \"No, because families with children are allowed to purchase units in the building without balconies.\",\n \"F\": \"No, because the association has the right to approve all sales.\",\n \"G\": \"No, because the amendment is based on legitimate safety issues.\"\n }\n}
input
question.json
{
"question": "A condominium development consists of two buildings, one with balconies attached to each unit, and one withno balconies. For safety concerns, the condominium association amended the covenants and restrictions toprohibit future sales of balcony units to families with minor children. The amendment did not affect families withchildren already living in balcony units. The amendment was promptly recorded. The condominium associationhad a valid covenant providing that all sales had to be approved by the association. Subsequent to the effective date of the amendment, the owner of a balcony unit contracted to sell it to a familywith minor children. Before the closing, the association told the buyers that because they had minor children,they could not buy the unit. The association further told the buyers that numerous units were available in thebuilding without balconies. After receiving this notification, the buyers complained to a fair housing agency, claiming that the amendmentwas unenforceable because it violated federal fair housing laws. Is there reasonable cause to believe that a violation has occurred?",
"options": {
"A": "Yes, because families with children are already living in units with balconies.",
"B": "No, because the amendment was promptly recorded and is legally binding.",
"C": "Yes, because families with children cannot be segregated within the condominium development.",
"D": "No, because the association is acting in the best interests of child safety.",
"E": "No, because families with children are allowed to purchase units in the building without balconies.",
"F": "No, because the association has the right to approve all sales.",
"G": "No, because the amendment is based on legitimate safety issues."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_097/ files.
Scored by a judge this pinned commit does not carry.
▸case_098Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Forced convective heat transfer occurs when water is flowing at a rate of 1.27 ft^3/hr through a pipe, 0.0492 ft dia and 3.28 ft long, at 140°F. The temperature of water at the inlet and outlet are 68°F and 86°F, respectively. Assuming laminar flow, calculate the heat transfer coefficient by a) Sieder and Tate's formula, and b) Hausen's formula.\",\n \"options\": {\n \"A\": \"Sieder and Tate's formula: 70.50 Btu/hr-ft^2-°F, Hausen's formula: 45.66 Btu/hr-ft^2-°F\",\n \"B\": \"Sieder and Tate's formula: 65.73 Btu/hr-ft^2-°F, Hausen's formula: 40.66 Btu/hr-ft^2-°F\",\n \"C\": \"Sieder and Tate's formula: 64.73 Btu/hr-ft^2-°F, Hausen's formula: 49.66 Btu/hr-ft^2-°F\",\n \"D\": \"Sieder and Tate's formula: 60.73 Btu/hr-ft^2-°F, Hausen's formula: 52.66 Btu/hr-ft^2-°F\"\n }\n}
input
question.json
{
"question": "Forced convective heat transfer occurs when water is flowing at a rate of 1.27 ft^3/hr through a pipe, 0.0492 ft dia and 3.28 ft long, at 140°F. The temperature of water at the inlet and outlet are 68°F and 86°F, respectively. Assuming laminar flow, calculate the heat transfer coefficient by a) Sieder and Tate's formula, and b) Hausen's formula.",
"options": {
"A": "Sieder and Tate's formula: 70.50 Btu/hr-ft^2-°F, Hausen's formula: 45.66 Btu/hr-ft^2-°F",
"B": "Sieder and Tate's formula: 65.73 Btu/hr-ft^2-°F, Hausen's formula: 40.66 Btu/hr-ft^2-°F",
"C": "Sieder and Tate's formula: 64.73 Btu/hr-ft^2-°F, Hausen's formula: 49.66 Btu/hr-ft^2-°F",
"D": "Sieder and Tate's formula: 60.73 Btu/hr-ft^2-°F, Hausen's formula: 52.66 Btu/hr-ft^2-°F"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_098/ files.
Scored by a judge this pinned commit does not carry.
▸case_099Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A husband and wife were shopping at a local department store when the wife saw a dress that she especially liked. The wife, who weighed more than 300 pounds, was unable to find a size large enough to fit her. She then saw a store clerk and asked if the store carried the dress in her size. The clerk looked at the wife and said, \\\"You look like a hippopotamus, and I'm sorry, but we don't carry this dress in the hippo size. \\\" Another customer in the store overheard the comment and began to laugh. The wife became very upset and hurried out of the store. If the wife asserts a tort action against the department store based upon the clerk's actions, the wife will\",\n \"options\": {\n \"A\": \"win, because the store does not carry clothes in her size.\",\n \"B\": \"win, because the clerk's conduct was extreme and outrageous.\",\n \"C\": \"win, because the clerk's actions caused her emotional distress.\",\n \"D\": \"lose, because the wife did not suffer any physical harm.\",\n \"E\": \"lose, because the clerk was merely statingan opinion.\",\n \"F\": \"win, because the comment was intended to ridicule and humiliate her.\",\n \"G\": \"win, because the statement was overheard byanother customer.\",\n \"H\": \"lose, because the comment was not intended to cause harm.\",\n \"I\": \"lose, because the wife only suffered hurt feelings.\",\n \"J\": \"lose, because the comment was made in a public place.\"\n }\n}
input
question.json
{
"question": "A husband and wife were shopping at a local department store when the wife saw a dress that she especially liked. The wife, who weighed more than 300 pounds, was unable to find a size large enough to fit her. She then saw a store clerk and asked if the store carried the dress in her size. The clerk looked at the wife and said, \"You look like a hippopotamus, and I'm sorry, but we don't carry this dress in the hippo size. \" Another customer in the store overheard the comment and began to laugh. The wife became very upset and hurried out of the store. If the wife asserts a tort action against the department store based upon the clerk's actions, the wife will",
"options": {
"A": "win, because the store does not carry clothes in her size.",
"B": "win, because the clerk's conduct was extreme and outrageous.",
"C": "win, because the clerk's actions caused her emotional distress.",
"D": "lose, because the wife did not suffer any physical harm.",
"E": "lose, because the clerk was merely statingan opinion.",
"F": "win, because the comment was intended to ridicule and humiliate her.",
"G": "win, because the statement was overheard byanother customer.",
"H": "lose, because the comment was not intended to cause harm.",
"I": "lose, because the wife only suffered hurt feelings.",
"J": "lose, because the comment was made in a public place."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_099/ files.
Scored by a judge this pinned commit does not carry.
▸case_100Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Find the Laplace transform L{g(t)}, where g(t)= 0,t < 4 and= (t - 4)^2,t \\\\geq 4\",\n \"options\": {\n \"A\": \"[(2e^-4s) / s^3]\",\n \"B\": \"2 / s^3\",\n \"C\": \"(2 / s^2) * e^-4s\",\n \"D\": \"4 / s^3\",\n \"E\": \"[(2e^-4s) / s^4]\",\n \"F\": \"(2 / s^3) * e^-s\",\n \"G\": \"e^-4s\",\n \"H\": \"[(e^-4s) / s^2]\",\n \"I\": \"(2 / s) * e^-4s\",\n \"J\": \"(t^2 - 8t + 16) / s\"\n }\n}
input
question.json
{
"question": "Find the Laplace transform L{g(t)}, where g(t)= 0,t < 4 and= (t - 4)^2,t \\geq 4",
"options": {
"A": "[(2e^-4s) / s^3]",
"B": "2 / s^3",
"C": "(2 / s^2) * e^-4s",
"D": "4 / s^3",
"E": "[(2e^-4s) / s^4]",
"F": "(2 / s^3) * e^-s",
"G": "e^-4s",
"H": "[(e^-4s) / s^2]",
"I": "(2 / s) * e^-4s",
"J": "(t^2 - 8t + 16) / s"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_100/ files.
Scored by a judge this pinned commit does not carry.
▸case_101Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A state was facing a fiscal problem. In order to raise funds, the state decided to consider the creation of a state postal service that only its residents may use for intrastate mail. The price for stamps printed by the state postal service would be half the cost of stamps printed by the U. S. Postal Service and would always be guaranteed to arrive within 24 hours. In addition, any employees of the state would be able to send their mail for free through the state postal service. Which of the following is the strongest constitutional argument against the proposed legislation?\",\n \"options\": {\n \"A\": \"It goes against the Supremacy Clause as it conflicts with federal laws governing postal services.\",\n \"B\": \"It is a violation of the Tenth Amendment as it encroaches on the powers reserved for the federal government.\",\n \"C\": \"It interferes with the sovereign autonomy of the U. S. Postal Service to operate as an independent federal agency.\",\n \"D\": \"It denies citizens of the state who are not state government employees their rights under the privileges or immunities clause of the Fourteenth Amendment.\",\n \"E\": \"It infringes on the First Amendment rights of individuals by limiting their choice of postal service.\",\n \"F\": \"It constitutes a violation of the commerce clause, as it interferes with interstate commerce.\",\n \"G\": \"It infringes on the rights of non-residents who might want to use the service.\",\n \"H\": \"It constitutes a denial of equal protection of the laws, because employees of the state receive personal benefits that are not enjoyed by employees of other states.\",\n \"I\": \"The negative implications that flow from the delegation to Congress of the power to establish a post office prohibit such a state postal service.\"\n }\n}
input
question.json
{
"question": "A state was facing a fiscal problem. In order to raise funds, the state decided to consider the creation of a state postal service that only its residents may use for intrastate mail. The price for stamps printed by the state postal service would be half the cost of stamps printed by the U. S. Postal Service and would always be guaranteed to arrive within 24 hours. In addition, any employees of the state would be able to send their mail for free through the state postal service. Which of the following is the strongest constitutional argument against the proposed legislation?",
"options": {
"A": "It goes against the Supremacy Clause as it conflicts with federal laws governing postal services.",
"B": "It is a violation of the Tenth Amendment as it encroaches on the powers reserved for the federal government.",
"C": "It interferes with the sovereign autonomy of the U. S. Postal Service to operate as an independent federal agency.",
"D": "It denies citizens of the state who are not state government employees their rights under the privileges or immunities clause of the Fourteenth Amendment.",
"E": "It infringes on the First Amendment rights of individuals by limiting their choice of postal service.",
"F": "It constitutes a violation of the commerce clause, as it interferes with interstate commerce.",
"G": "It infringes on the rights of non-residents who might want to use the service.",
"H": "It constitutes a denial of equal protection of the laws, because employees of the state receive personal benefits that are not enjoyed by employees of other states.",
"I": "The negative implications that flow from the delegation to Congress of the power to establish a post office prohibit such a state postal service."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_101/ files.
Scored by a judge this pinned commit does not carry.
▸case_102Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A nightclub owner applied for a required zoning permit to open a nude-dancing nightclub in the theater district of a city. An organization of influential city residents began an intensive lobbying effort to persuade the city council to deny the owner a permit to operate any type of nude- dancing facility at any time or in any place in the city. The owner has sued the city in an appropriate federal court, seeking an injunction that would prohibit the city council from considering the organization's views, on the ground that if the organization is successful in its lobbying efforts, the owner's First and Fourteenth Amendment rights would be violated. The city has moved to dismiss the action. Should the court dismiss the owner's action? \",\n \"options\": {\n \"A\": \"Yes, because the city council has the right to consider the views of its citizens.\",\n \"B\": \"No, because the city council's consideration of the organization's views is biased and unfair.\",\n \"C\": \"Yes, because the organization's lobbying efforts are a lawful expression of their views.\",\n \"D\": \"No, because the organization's lobbying efforts infringe on the owner's right to conduct business.\",\n \"E\": \"No, because the organization does not seek a reasonable time, place, and manner regulation of nude dancing, but instead seeks a total ban on the owner's opening any type of nude-dancing facility at any time or in any place in the city.\",\n \"F\": \"Yes, because the action is not ripe.\",\n \"G\": \"No, because the owner has a constitutional right to free expression, which includes operating a nude-dancing facility.\",\n \"H\": \"Yes, because the First and Fourteenth Amendments do not protect obscenity, and nude dancing is obscene.\",\n \"I\": \"Yes, because the owner's lawsuit interferes with the city council's decision-making process.\",\n \"J\": \"No, because nude dancing is symbolic speech and is therefore protected by the First and Fourteenth Amendments.\"\n }\n}
input
question.json
{
"question": "A nightclub owner applied for a required zoning permit to open a nude-dancing nightclub in the theater district of a city. An organization of influential city residents began an intensive lobbying effort to persuade the city council to deny the owner a permit to operate any type of nude- dancing facility at any time or in any place in the city. The owner has sued the city in an appropriate federal court, seeking an injunction that would prohibit the city council from considering the organization's views, on the ground that if the organization is successful in its lobbying efforts, the owner's First and Fourteenth Amendment rights would be violated. The city has moved to dismiss the action. Should the court dismiss the owner's action? ",
"options": {
"A": "Yes, because the city council has the right to consider the views of its citizens.",
"B": "No, because the city council's consideration of the organization's views is biased and unfair.",
"C": "Yes, because the organization's lobbying efforts are a lawful expression of their views.",
"D": "No, because the organization's lobbying efforts infringe on the owner's right to conduct business.",
"E": "No, because the organization does not seek a reasonable time, place, and manner regulation of nude dancing, but instead seeks a total ban on the owner's opening any type of nude-dancing facility at any time or in any place in the city.",
"F": "Yes, because the action is not ripe.",
"G": "No, because the owner has a constitutional right to free expression, which includes operating a nude-dancing facility.",
"H": "Yes, because the First and Fourteenth Amendments do not protect obscenity, and nude dancing is obscene.",
"I": "Yes, because the owner's lawsuit interferes with the city council's decision-making process.",
"J": "No, because nude dancing is symbolic speech and is therefore protected by the First and Fourteenth Amendments."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_102/ files.
Scored by a judge this pinned commit does not carry.
▸case_103Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"On June 1, an appliance manufacturer telephoned a supplier to determine whether the supplier could provide 300 washing machine motors of a particular model by October 1. The supplier offered to do so at a price of $300 per motor (a total price of $90,000). The manufacturer's representative said, \\\"Deal.\\\" The next day the manufacturer's representative sent the supplier an unsigned note on company letterhead that stated, \\\"I am happy that you are going to supply us with the motors. I will call you soon to talk about another order.\\\" The manufacturer then sent catalogs to its regular customers advertising washing machines that included the specified motors. The manufacturer did not hear from the supplier until July 1, when the supplier called to say that it would be unable to supply the motors because it was no longer carrying that model. At that time, the manufacturer had received no orders for the machines with the specified motors. The manufacturer sued the supplier for breach of contract, and the supplier raised the statute of frauds as a defense. Is the supplier's statute of frauds defense likely to succeed?\",\n \"options\": {\n \"A\": \"Yes, because the manufacturer had not received any orders for the machines with the specified motors at the time of the supplier's refusal.\",\n \"B\": \"No, because the manufacturer's representative verbally agreed to the deal over the phone.\",\n \"C\": \"No, because the supplier failed to object to the contents of the note sent by the manufacturer.\",\n \"D\": \"Yes, because the supplier was no longer carrying the model of the motor at the time of the call.\",\n \"E\": \"Yes, because there is no writing that contains the quantity term of the contract.\",\n \"F\": \"Yes, because the manufacturer's note failed to contain a signature.\",\n \"G\": \"No, because the manufacturer's representative sent the note on company letterhead.\",\n \"H\": \"No, because the supplier did not inform the manufacturer about not being able to supply the motors in time.\",\n \"I\": \"Yes, because the supplier never agreed to the contract in writing.\",\n \"J\": \"No, because the manufacturer distributed the catalogs in reliance on the contract, making the contract enforceable under the statute of frauds.\"\n }\n}
input
question.json
{
"question": "On June 1, an appliance manufacturer telephoned a supplier to determine whether the supplier could provide 300 washing machine motors of a particular model by October 1. The supplier offered to do so at a price of $300 per motor (a total price of $90,000). The manufacturer's representative said, \"Deal.\" The next day the manufacturer's representative sent the supplier an unsigned note on company letterhead that stated, \"I am happy that you are going to supply us with the motors. I will call you soon to talk about another order.\" The manufacturer then sent catalogs to its regular customers advertising washing machines that included the specified motors. The manufacturer did not hear from the supplier until July 1, when the supplier called to say that it would be unable to supply the motors because it was no longer carrying that model. At that time, the manufacturer had received no orders for the machines with the specified motors. The manufacturer sued the supplier for breach of contract, and the supplier raised the statute of frauds as a defense. Is the supplier's statute of frauds defense likely to succeed?",
"options": {
"A": "Yes, because the manufacturer had not received any orders for the machines with the specified motors at the time of the supplier's refusal.",
"B": "No, because the manufacturer's representative verbally agreed to the deal over the phone.",
"C": "No, because the supplier failed to object to the contents of the note sent by the manufacturer.",
"D": "Yes, because the supplier was no longer carrying the model of the motor at the time of the call.",
"E": "Yes, because there is no writing that contains the quantity term of the contract.",
"F": "Yes, because the manufacturer's note failed to contain a signature.",
"G": "No, because the manufacturer's representative sent the note on company letterhead.",
"H": "No, because the supplier did not inform the manufacturer about not being able to supply the motors in time.",
"I": "Yes, because the supplier never agreed to the contract in writing.",
"J": "No, because the manufacturer distributed the catalogs in reliance on the contract, making the contract enforceable under the statute of frauds."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_103/ files.
Scored by a judge this pinned commit does not carry.
▸case_104Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A 250-kw 230-volt compound generator is delivering 800 amp at 230 volts. The shunt-field current is 12 amp. The armature resistance is 0.007 ohm, and the series-field resistance is 0.002 ohm. The stray power at this load is 5,500 watts. The generator is connected long-shunt. Determine generator efficiency at this load.\",\n \"options\": {\n \"A\": \"95.0%\",\n \"B\": \"91.7%\",\n \"C\": \"89.5%\",\n \"D\": \"93.5%\",\n \"E\": \"90.0%\",\n \"F\": \"87.3%\",\n \"G\": \"86.0%\",\n \"H\": \"94.2%\",\n \"I\": \"88.0%\",\n \"J\": \"92.0%\"\n }\n}
input
question.json
{
"question": "A 250-kw 230-volt compound generator is delivering 800 amp at 230 volts. The shunt-field current is 12 amp. The armature resistance is 0.007 ohm, and the series-field resistance is 0.002 ohm. The stray power at this load is 5,500 watts. The generator is connected long-shunt. Determine generator efficiency at this load.",
"options": {
"A": "95.0%",
"B": "91.7%",
"C": "89.5%",
"D": "93.5%",
"E": "90.0%",
"F": "87.3%",
"G": "86.0%",
"H": "94.2%",
"I": "88.0%",
"J": "92.0%"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_104/ files.
Scored by a judge this pinned commit does not carry.
▸case_105Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Use the Laplace transform of f(t) = e^kt,(1) where k is a complex constant of the form k = Re{k} + i Im{k} with Re{k} the real part of k, Im{k} the imaginary part of k, and i \\\\equiv \\\\surd(- 1), to find the Laplace transforms of f(t) = cosh kt, sinh kt, cos kt, and sin kt.\",\n \"options\": {\n \"A\": \"L{cosh kt} = [k / (s^2 - k^2)], L{sinh kt} = [s / (s^2 + k^2)], L{cos kt} = [k / (s^2 + k^2)], L{sin kt} = [s / (s^2 - k^2)]\",\n \"B\": \"L{cosh kt} = [2s / (s^2 - k^2)], L{sinh kt} = [2k / (s^2 - k^2)], L{cos kt} = [2s / (s^2 + k^2)], L{sin kt} = [2k / (s^2 + k^2)]\",\n \"C\": \"L{cosh kt} = [s / (s^2 + k^2)], L{sinh kt} = [k / (s^2 - k^2)], L{cos kt} = [s / (s^2 - k^2)], L{sin kt} = [k / (s^2 + k^2)]\",\n \"D\": \"L{cosh kt} = [k / (s^2 + k^2)], L{sinh kt} = [s / (s^2 + k^2)], L{cos kt} = [k / (s^2 - k^2)], L{sin kt} = [s / (s^2 + k^2)]\",\n \"E\": \"L{cosh kt} = [s / (s^2 - k^2)], L{sinh kt} = [2k / (s^2 + k^2)], L{cos kt} = [2s / (s^2 - k^2)], L{sin kt} = [s / (s^2 + 2k^2)]\",\n \"F\": \"L{cosh kt} = [s / (s^2 + k^2)], L{sinh kt} = [k / (s^2 + k^2)], L{cos kt} = [s / (s^2 - k^2)], L{sin kt} = [k / (s^2 - k^2)]\",\n \"G\": \"L{cosh kt} = [s / (s^2 + 2k^2)], L{sinh kt} = [k / (s^2 + 2k^2)], L{cos kt} = [s / (s^2 - 2k^2)], L{sin kt} = [k / (s^2 - 2k^2)]\",\n \"H\": \"L{cosh kt} = [s / (s^2 - 2k^2)], L{sinh kt} = [k / (s^2 + 2k^2)], L{cos kt} = [s / (s^2 + 2k^2)], L{sin kt} = [k / (s^2 - 2k^2)]\",\n \"I\": \"L{cosh kt} = [k / (s^2 + k^2)], L{sinh kt} = [s / (s^2 - k^2)], L{cos kt} = [s / (s^2 - k^2)], L{sin kt} = [k / (s^2 - k^2)]\",\n \"J\": \"L{cosh kt} = [s / (s^2 - k^2)], L{sinh kt} = [k / (s^2 - k^2)], L{cos kt} = [s / (s^2 + k^2)], L{sin kt} = [k / (s^2 + k^2)]\"\n }\n}
input
question.json
{
"question": "Use the Laplace transform of f(t) = e^kt,(1) where k is a complex constant of the form k = Re{k} + i Im{k} with Re{k} the real part of k, Im{k} the imaginary part of k, and i \\equiv \\surd(- 1), to find the Laplace transforms of f(t) = cosh kt, sinh kt, cos kt, and sin kt.",
"options": {
"A": "L{cosh kt} = [k / (s^2 - k^2)], L{sinh kt} = [s / (s^2 + k^2)], L{cos kt} = [k / (s^2 + k^2)], L{sin kt} = [s / (s^2 - k^2)]",
"B": "L{cosh kt} = [2s / (s^2 - k^2)], L{sinh kt} = [2k / (s^2 - k^2)], L{cos kt} = [2s / (s^2 + k^2)], L{sin kt} = [2k / (s^2 + k^2)]",
"C": "L{cosh kt} = [s / (s^2 + k^2)], L{sinh kt} = [k / (s^2 - k^2)], L{cos kt} = [s / (s^2 - k^2)], L{sin kt} = [k / (s^2 + k^2)]",
"D": "L{cosh kt} = [k / (s^2 + k^2)], L{sinh kt} = [s / (s^2 + k^2)], L{cos kt} = [k / (s^2 - k^2)], L{sin kt} = [s / (s^2 + k^2)]",
"E": "L{cosh kt} = [s / (s^2 - k^2)], L{sinh kt} = [2k / (s^2 + k^2)], L{cos kt} = [2s / (s^2 - k^2)], L{sin kt} = [s / (s^2 + 2k^2)]",
"F": "L{cosh kt} = [s / (s^2 + k^2)], L{sinh kt} = [k / (s^2 + k^2)], L{cos kt} = [s / (s^2 - k^2)], L{sin kt} = [k / (s^2 - k^2)]",
"G": "L{cosh kt} = [s / (s^2 + 2k^2)], L{sinh kt} = [k / (s^2 + 2k^2)], L{cos kt} = [s / (s^2 - 2k^2)], L{sin kt} = [k / (s^2 - 2k^2)]",
"H": "L{cosh kt} = [s / (s^2 - 2k^2)], L{sinh kt} = [k / (s^2 + 2k^2)], L{cos kt} = [s / (s^2 + 2k^2)], L{sin kt} = [k / (s^2 - 2k^2)]",
"I": "L{cosh kt} = [k / (s^2 + k^2)], L{sinh kt} = [s / (s^2 - k^2)], L{cos kt} = [s / (s^2 - k^2)], L{sin kt} = [k / (s^2 - k^2)]",
"J": "L{cosh kt} = [s / (s^2 - k^2)], L{sinh kt} = [k / (s^2 - k^2)], L{cos kt} = [s / (s^2 + k^2)], L{sin kt} = [k / (s^2 + k^2)]"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_105/ files.
Scored by a judge this pinned commit does not carry.
▸case_106Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A college student and her boyfriend lived together in a one-bedroom apartment in Los Angeles. They were engaged to be married when they purchased an undeveloped parcel of land on a hillside overlooking the Pacific. The deed, which was properly executed and recorded, named the student and the boyfriend as grantees, \\\"not as tenants in common but as joint tenants with right of survivorship. \\\"Thereafter, the boyfriend, who was experiencing financial difficulties, offered to sell the property to his co-worker. Without the student's knowledge, the boyfriend executed a deed that purported to convey the hillside property to the co-worker in fee simple. The boyfriend signed his name and forged the student's name. He then delivered the deed of conveyance to the co-worker, who paid the boyfriend $150,000, which was the fair market value of the property. The co-worker immediately recorded the deed received from the boyfriend. The common law joint tenancy is unmodified by statute. Title to the property in question is now in\",\n \"options\": {\n \"A\": \"the co-worker, the student and the boyfriend as joint tenants with right of survivorship.\",\n \"B\": \"the co-worker as owner in fee simple.\",\n \"C\": \"the boyfriend and the student as joint tenants with right of survivorship.\",\n \"D\": \"the co-worker and the boyfriend as tenants in common.\",\n \"E\": \"the co-worker and the student as joint tenants with right of survivorship.\",\n \"F\": \"the co-worker and the boyfriend as joint tenants with right of survivorship.\",\n \"G\": \"the co-worker and the student as tenants in common.\",\n \"H\": \"the co-worker, the student and the boyfriend as tenants in common.\",\n \"I\": \"the boyfriend as owner in fee simple.\",\n \"J\": \"the student as owner in fee simple.\"\n }\n}
input
question.json
{
"question": "A college student and her boyfriend lived together in a one-bedroom apartment in Los Angeles. They were engaged to be married when they purchased an undeveloped parcel of land on a hillside overlooking the Pacific. The deed, which was properly executed and recorded, named the student and the boyfriend as grantees, \"not as tenants in common but as joint tenants with right of survivorship. \"Thereafter, the boyfriend, who was experiencing financial difficulties, offered to sell the property to his co-worker. Without the student's knowledge, the boyfriend executed a deed that purported to convey the hillside property to the co-worker in fee simple. The boyfriend signed his name and forged the student's name. He then delivered the deed of conveyance to the co-worker, who paid the boyfriend $150,000, which was the fair market value of the property. The co-worker immediately recorded the deed received from the boyfriend. The common law joint tenancy is unmodified by statute. Title to the property in question is now in",
"options": {
"A": "the co-worker, the student and the boyfriend as joint tenants with right of survivorship.",
"B": "the co-worker as owner in fee simple.",
"C": "the boyfriend and the student as joint tenants with right of survivorship.",
"D": "the co-worker and the boyfriend as tenants in common.",
"E": "the co-worker and the student as joint tenants with right of survivorship.",
"F": "the co-worker and the boyfriend as joint tenants with right of survivorship.",
"G": "the co-worker and the student as tenants in common.",
"H": "the co-worker, the student and the boyfriend as tenants in common.",
"I": "the boyfriend as owner in fee simple.",
"J": "the student as owner in fee simple."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_106/ files.
Scored by a judge this pinned commit does not carry.
▸case_107Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"An avowed corrimunist was elected vice president of a union. A senator, in his investigation of communist infiltration of national labor unions, found a provision in a statute passed by Congress, which makes it a crime for a member of the communist party to act as an official of a labor union. After a subsequent legislative hearing, the communist is dismissed from his position by the union. Which of the following most accurately summarizes the applicable rule of constitutional law regarding the aforementioned provision of the statute?\",\n \"options\": {\n \"A\": \"The statutory provision is a violation of the person's Eighth Amendment right against cruel and unusual punishment.\",\n \"B\": \"The statutory provision is a violation of the person's Sixth Amendment right to a fair trial.\",\n \"C\": \"The statutory provision in the act is a form of legislative punishment violative of the Constitution as a bill of attainder.\",\n \"D\": \"Making it a crime for a Communist to hold a union office is a suspect classification, which violates the equal protection clause.\",\n \"E\": \"The statutory prohibition is a reasonable method of discrimination since the benefit to the public outweighs the injury or restrictions that would be inflicted upon the person.\",\n \"F\": \"The statute is valid as it is a reasonable restriction on the First Amendment rights of an individual in the interest of national security.\",\n \"G\": \"The statutory provision is unconstitutional as it violates the person's right to due process under the Fifth Amendment.\",\n \"H\": \"The statutory provision is a form of unconstitutional prior restraint on a person's First Amendment right of free association.\",\n \"I\": \"The statutory prohibition is a valid exercise of Congress' power to regulate interstate commerce.\",\n \"J\": \"The statute is unconstitutional because it infringes upon the individual's right to political affiliation, protected under the First Amendment.\"\n }\n}
input
question.json
{
"question": "An avowed corrimunist was elected vice president of a union. A senator, in his investigation of communist infiltration of national labor unions, found a provision in a statute passed by Congress, which makes it a crime for a member of the communist party to act as an official of a labor union. After a subsequent legislative hearing, the communist is dismissed from his position by the union. Which of the following most accurately summarizes the applicable rule of constitutional law regarding the aforementioned provision of the statute?",
"options": {
"A": "The statutory provision is a violation of the person's Eighth Amendment right against cruel and unusual punishment.",
"B": "The statutory provision is a violation of the person's Sixth Amendment right to a fair trial.",
"C": "The statutory provision in the act is a form of legislative punishment violative of the Constitution as a bill of attainder.",
"D": "Making it a crime for a Communist to hold a union office is a suspect classification, which violates the equal protection clause.",
"E": "The statutory prohibition is a reasonable method of discrimination since the benefit to the public outweighs the injury or restrictions that would be inflicted upon the person.",
"F": "The statute is valid as it is a reasonable restriction on the First Amendment rights of an individual in the interest of national security.",
"G": "The statutory provision is unconstitutional as it violates the person's right to due process under the Fifth Amendment.",
"H": "The statutory provision is a form of unconstitutional prior restraint on a person's First Amendment right of free association.",
"I": "The statutory prohibition is a valid exercise of Congress' power to regulate interstate commerce.",
"J": "The statute is unconstitutional because it infringes upon the individual's right to political affiliation, protected under the First Amendment."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_107/ files.
Scored by a judge this pinned commit does not carry.
▸case_108Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A man borrowed $150,000 from a bank to remodel his home and executed a promissory note agreeing to repay the loan over a 10-year period. The loan was secured by a mortgage on the home. The bank promptly recorded the mortgage, which was the only lien on the home at that time. Several months later, the man borrowed $40,000 from his mother in order to purchase a new truck and gave his mother a mortgage on the home to secure repayment of the debt. The mother promptly recorded the mortgage. The man later lost his job and was struggling to make loan payments to both the bank and his mother. To accommodate the man's financial situation, the bank extended the amortization period of its loan to substantially reduce the amount of each monthly payment. The bank did not alter the interest rate or increase the principal amount of the loan. At the time of the modification of the bank loan, the man was not in default in his payments to his mother. Neither the bank nor the man informed the mother of the modification of the bank loan. After the man later missed five payments to his mother, she commenced a foreclosure action. While the action was pending, the mother learned of the bank's loan modification. The mother asserted that her mortgage had become a first lien on the home and that the bank's mortgage had become a second lien. Is the mother's assertion correct?\",\n \"options\": {\n \"A\": \"No, because the bank's loan modification was not detrimental to the rights of the mother.\",\n \"B\": \"Yes, because the man failed to inform his mother about the bank's loan modification.\",\n \"C\": \"Yes, because the bank's loan modification was made without the mother's prior consent.\",\n \"D\": \"No, because the mother did not take action when she learned about the bank's loan modification.\",\n \"E\": \"No, because the man's obligation to repay his mother was not affected by the bank's loan modification.\",\n \"F\": \"No, because the man was not in default in his payments to his mother at the time of the bank's loan modification.\",\n \"G\": \"Yes, because the bank's loan modification was material.\",\n \"H\": \"Yes, because the bank's loan modification extended the repayment period.\",\n \"I\": \"Yes, because the man was struggling to make his payments to his mother.\"\n }\n}
input
question.json
{
"question": "A man borrowed $150,000 from a bank to remodel his home and executed a promissory note agreeing to repay the loan over a 10-year period. The loan was secured by a mortgage on the home. The bank promptly recorded the mortgage, which was the only lien on the home at that time. Several months later, the man borrowed $40,000 from his mother in order to purchase a new truck and gave his mother a mortgage on the home to secure repayment of the debt. The mother promptly recorded the mortgage. The man later lost his job and was struggling to make loan payments to both the bank and his mother. To accommodate the man's financial situation, the bank extended the amortization period of its loan to substantially reduce the amount of each monthly payment. The bank did not alter the interest rate or increase the principal amount of the loan. At the time of the modification of the bank loan, the man was not in default in his payments to his mother. Neither the bank nor the man informed the mother of the modification of the bank loan. After the man later missed five payments to his mother, she commenced a foreclosure action. While the action was pending, the mother learned of the bank's loan modification. The mother asserted that her mortgage had become a first lien on the home and that the bank's mortgage had become a second lien. Is the mother's assertion correct?",
"options": {
"A": "No, because the bank's loan modification was not detrimental to the rights of the mother.",
"B": "Yes, because the man failed to inform his mother about the bank's loan modification.",
"C": "Yes, because the bank's loan modification was made without the mother's prior consent.",
"D": "No, because the mother did not take action when she learned about the bank's loan modification.",
"E": "No, because the man's obligation to repay his mother was not affected by the bank's loan modification.",
"F": "No, because the man was not in default in his payments to his mother at the time of the bank's loan modification.",
"G": "Yes, because the bank's loan modification was material.",
"H": "Yes, because the bank's loan modification extended the repayment period.",
"I": "Yes, because the man was struggling to make his payments to his mother."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_108/ files.
Scored by a judge this pinned commit does not carry.
▸case_109Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Calculate the drag force acting on a 3 ft wide by 4 ft long planesurface when water flows over it at a free-stream velocityof 5 ft/s. The laminar boundary layer region ends at aReynolds number of 1 × 10^5.\",\n \"options\": {\n \"A\": \"1.35lb_f\",\n \"B\": \"1.15lb_f\",\n \"C\": \"1.05 lb_f\",\n \"D\": \"1.25 lb_f\",\n \"E\": \"1.50 lb_f\",\n \"F\": \"0.85lb_f\",\n \"G\": \"0.65 lb_f\",\n \"H\": \"0.75 lb_f\",\n \"I\": \"0.97lb_f\",\n \"J\": \"0.55 lb_f\"\n }\n}
input
question.json
{
"question": "Calculate the drag force acting on a 3 ft wide by 4 ft long planesurface when water flows over it at a free-stream velocityof 5 ft/s. The laminar boundary layer region ends at aReynolds number of 1 × 10^5.",
"options": {
"A": "1.35lb_f",
"B": "1.15lb_f",
"C": "1.05 lb_f",
"D": "1.25 lb_f",
"E": "1.50 lb_f",
"F": "0.85lb_f",
"G": "0.65 lb_f",
"H": "0.75 lb_f",
"I": "0.97lb_f",
"J": "0.55 lb_f"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_109/ files.
Scored by a judge this pinned commit does not carry.
▸case_110Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"An inventor developed a prototype of an automobile. engine that she believed was going to revolutionize the automotive industry because of the engine's fuel-efficient properties. As a wedding present to the inventor's brother, the inventor wrote on a wedding card she gave to her brother:\\\"I hereby assign to you 20 percent of the net profits realized from the production and distribution of my automobile engine within the first year of production. \\\"The inventor then borrowed money from creditors and began to promote her automobile engine. She was contacted by a major automobile manufacturer about the possibility of using the engine in a new line of cars the manufacturer intended to release during the next calendar year. While on her way to meet with the representatives of the manufacturer, the inventor was killed in an automobile collision. The manufacturer then contracted with the executor of the inventor's estate to license the use of the automobile engine design for a 10-year period for $500,000, payable to the estate. The creditors from whom the inventor borrowed the money to promote her automobile engine have now attached the proceeds from the licensing agreement, which have so far been paid into the estate. The brother also asserts his assigned rights to a share of the proceeds. In subsequent actions by the creditors and the brother, the court will most probably hold in favor of\",\n \"options\": {\n \"A\": \"the creditors, because the agreement with the manufacturer was made after the assignment to the brother.\",\n \"B\": \"the brother, because the inventor's assignment was recorded in a written document.\",\n \"C\": \"the brother, because the assignment was given as a gift and thus cannot be contested by the creditors.\",\n \"D\": \"the creditors, because the inventor's assignment to the brother was unenforceable as a defective assignment.\",\n \"E\": \"the creditors, because the rights of creditors/lienors are superior to those of a donee beneficiary.\",\n \"F\": \"the creditors, because the inventor's assignment to the brother was not legally binding.\",\n \"G\": \"the brother, because the inventor's assignment to him was made prior to the agreement with the manufacturer.\",\n \"H\": \"the brother, because any proceeds of the assigned right received by the assignor thereafter are held in constructive trust for the assignee.\",\n \"I\": \"the brother, because the rights of an assignee are superior to a lien against the assignor subsequently obtained by legallequitable proceedings.\",\n \"J\": \"the creditors, because they provided financial support to the inventor's project.\"\n }\n}
input
question.json
{
"question": "An inventor developed a prototype of an automobile. engine that she believed was going to revolutionize the automotive industry because of the engine's fuel-efficient properties. As a wedding present to the inventor's brother, the inventor wrote on a wedding card she gave to her brother:\"I hereby assign to you 20 percent of the net profits realized from the production and distribution of my automobile engine within the first year of production. \"The inventor then borrowed money from creditors and began to promote her automobile engine. She was contacted by a major automobile manufacturer about the possibility of using the engine in a new line of cars the manufacturer intended to release during the next calendar year. While on her way to meet with the representatives of the manufacturer, the inventor was killed in an automobile collision. The manufacturer then contracted with the executor of the inventor's estate to license the use of the automobile engine design for a 10-year period for $500,000, payable to the estate. The creditors from whom the inventor borrowed the money to promote her automobile engine have now attached the proceeds from the licensing agreement, which have so far been paid into the estate. The brother also asserts his assigned rights to a share of the proceeds. In subsequent actions by the creditors and the brother, the court will most probably hold in favor of",
"options": {
"A": "the creditors, because the agreement with the manufacturer was made after the assignment to the brother.",
"B": "the brother, because the inventor's assignment was recorded in a written document.",
"C": "the brother, because the assignment was given as a gift and thus cannot be contested by the creditors.",
"D": "the creditors, because the inventor's assignment to the brother was unenforceable as a defective assignment.",
"E": "the creditors, because the rights of creditors/lienors are superior to those of a donee beneficiary.",
"F": "the creditors, because the inventor's assignment to the brother was not legally binding.",
"G": "the brother, because the inventor's assignment to him was made prior to the agreement with the manufacturer.",
"H": "the brother, because any proceeds of the assigned right received by the assignor thereafter are held in constructive trust for the assignee.",
"I": "the brother, because the rights of an assignee are superior to a lien against the assignor subsequently obtained by legallequitable proceedings.",
"J": "the creditors, because they provided financial support to the inventor's project."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_110/ files.
Scored by a judge this pinned commit does not carry.
▸case_111Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"In a single throw of a single die, find the probability of obtaining either a 2 or a 5.\",\n \"options\": {\n \"A\": \"1/2\",\n \"B\": \"2/3\",\n \"C\": \"1/6\",\n \"D\": \"3/8\",\n \"E\": \"1/3\",\n \"F\": \"3/6\",\n \"G\": \"4/6\",\n \"H\": \"5/6\",\n \"I\": \"1/4\"\n }\n}
input
question.json
{
"question": "In a single throw of a single die, find the probability of obtaining either a 2 or a 5.",
"options": {
"A": "1/2",
"B": "2/3",
"C": "1/6",
"D": "3/8",
"E": "1/3",
"F": "3/6",
"G": "4/6",
"H": "5/6",
"I": "1/4"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_111/ files.
Scored by a judge this pinned commit does not carry.
▸case_112Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Copper normally solidifies at a temperature of 1356°K, but canexist in the super-cooled liquid form up to a temperature of1120°K, after which it starts solidifying spontaneously. Determine the entropy change for the solidification of copper atits temperature of spontaneous solidification (1120°K). Use the following data to solve the problem: Data forCu(1)\\\\rightleftharpoons Cu(s) CpCu(1)= 7.50 cal deg^-1 mole^-1 C_p Cu(s) = 5.41 + 1.50 × 10^-3T cal deg^-1 mole^-1 \\\\DeltaH^0 = - 3100 cal\",\n \"options\": {\n \"A\": \"- 2.50 cal deg^-1 mole^-1\",\n \"B\": \"- 2.09 cal deg^-1 mole^-1\",\n \"C\": \"- 2.23 cal deg^-1 mole^-1\",\n \"D\": \"- 1.95 cal deg^-1 mole^-1\",\n \"E\": \"- 2.28 cal deg^-1 mole^-1\",\n \"F\": \"- 3.10 cal deg^-1 mole^-1\",\n \"G\": \"3.00 cal deg^-1 mole^-1\",\n \"H\": \"2.09 cal deg^-1 mole^-1\",\n \"I\": \"0.046 cal deg^-1 mole^-1\",\n \"J\": \"- 1.75 cal deg^-1 mole^-1\"\n }\n}
input
question.json
{
"question": "Copper normally solidifies at a temperature of 1356°K, but canexist in the super-cooled liquid form up to a temperature of1120°K, after which it starts solidifying spontaneously. Determine the entropy change for the solidification of copper atits temperature of spontaneous solidification (1120°K). Use the following data to solve the problem: Data forCu(1)\\rightleftharpoons Cu(s) CpCu(1)= 7.50 cal deg^-1 mole^-1 C_p Cu(s) = 5.41 + 1.50 × 10^-3T cal deg^-1 mole^-1 \\DeltaH^0 = - 3100 cal",
"options": {
"A": "- 2.50 cal deg^-1 mole^-1",
"B": "- 2.09 cal deg^-1 mole^-1",
"C": "- 2.23 cal deg^-1 mole^-1",
"D": "- 1.95 cal deg^-1 mole^-1",
"E": "- 2.28 cal deg^-1 mole^-1",
"F": "- 3.10 cal deg^-1 mole^-1",
"G": "3.00 cal deg^-1 mole^-1",
"H": "2.09 cal deg^-1 mole^-1",
"I": "0.046 cal deg^-1 mole^-1",
"J": "- 1.75 cal deg^-1 mole^-1"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_112/ files.
Scored by a judge this pinned commit does not carry.
▸case_113Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Consider a horizontal pipe of 1.2 in. inside diameter and of roughness k = 0.002 ft. If water flows through it at 9.3 ft/s, calculate the pressure loss per 10 ft. of the pipe length. Kinematic viscosity of water ѵ = 0.93 × 10^-5 ft^2/s.\",\n \"options\": {\n \"A\": \"4.0 psi\",\n \"B\": \"4.5 psi\",\n \"C\": \"3.5 psi\",\n \"D\": \"3.0 psi\",\n \"E\": \"2.78 psi\",\n \"F\": \"1.8 psi\",\n \"G\": \"2.5 psi\",\n \"H\": \"3.2 psi\",\n \"I\": \"1.5 psi\",\n \"J\": \"2.0 psi\"\n }\n}
input
question.json
{
"question": "Consider a horizontal pipe of 1.2 in. inside diameter and of roughness k = 0.002 ft. If water flows through it at 9.3 ft/s, calculate the pressure loss per 10 ft. of the pipe length. Kinematic viscosity of water ѵ = 0.93 × 10^-5 ft^2/s.",
"options": {
"A": "4.0 psi",
"B": "4.5 psi",
"C": "3.5 psi",
"D": "3.0 psi",
"E": "2.78 psi",
"F": "1.8 psi",
"G": "2.5 psi",
"H": "3.2 psi",
"I": "1.5 psi",
"J": "2.0 psi"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_113/ files.
Scored by a judge this pinned commit does not carry.
▸case_114Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Calculate\\\\Deltah_vapof liquid oxygen if it boils at 90.15°K and has a vapor pressure of 2.5atmat 100°K.\",\n \"options\": {\n \"A\": \"1200 cal/g-mol\",\n \"B\": \"2100 cal/g-mol\",\n \"C\": \"1800 cal/g-mol\",\n \"D\": \"1750 cal/g-mol\",\n \"E\": \"1500 cal/g-mol\",\n \"F\": \"1900 cal/g-mol\",\n \"G\": \"1658 cal/g-mol\",\n \"H\": \"2000 cal/g-mol\",\n \"I\": \"1400 cal/g-mol\",\n \"J\": \"1550 cal/g-mol\"\n }\n}
input
question.json
{
"question": "Calculate\\Deltah_vapof liquid oxygen if it boils at 90.15°K and has a vapor pressure of 2.5atmat 100°K.",
"options": {
"A": "1200 cal/g-mol",
"B": "2100 cal/g-mol",
"C": "1800 cal/g-mol",
"D": "1750 cal/g-mol",
"E": "1500 cal/g-mol",
"F": "1900 cal/g-mol",
"G": "1658 cal/g-mol",
"H": "2000 cal/g-mol",
"I": "1400 cal/g-mol",
"J": "1550 cal/g-mol"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_114/ files.
Scored by a judge this pinned commit does not carry.
▸case_115Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A spherical shell of paraffin is contained between two spherical conductors. The inner sphere has a radius of 0.60 meter, and the inner radius of the outer sphere is 0.70 meter. (a) What is the capacitance of the space between the conductors? (b) What is the largest electric potential difference which may be applied to the system if dielectric breakdown is to be avoided? (c) If the electric potential difference of (b) could be established between the two spheres, how much energy would be stored in the electric field? (d) If the heat loss from a particular building under certain conditions is 40,000 Btu per hour, for how long would the energy in the electric field of (c) maintain the temperature in this building if this energy were released?\",\n \"options\": {\n \"A\": \"0.00085 microfarad, 60 × 10^6 volts, 68 × 10^3 joules, 6.0 seconds\",\n \"B\": \"0.00125 microfarad, 42 × 10^6 volts, 52 × 10^3 joules, 4.3 seconds\",\n \"C\": \"0.00108 microfarad, 48 × 10^6 volts, 54 × 10^3 joules, 4.6 seconds\",\n \"D\": \"0.00110 microfarad, 45 × 10^6 volts, 58 × 10^3 joules, 4.5 seconds\",\n \"E\": \"0.00103 microfarad, 44 × 10^6 volts, 56 × 10^3 joules, 4.8 seconds\",\n \"F\": \"0.00150 microfarad, 35 × 10^6 volts, 45 × 10^3 joules, 3.6 seconds\",\n \"G\": \"0.00120 microfarad, 40 × 10^6 volts, 50 × 10^3 joules, 4.0 seconds\",\n \"H\": \"0.00092 microfarad, 50 × 10^6 volts, 60 × 10^3 joules, 5.2 seconds\",\n \"I\": \"0.00140 microfarad, 38 × 10^6 volts, 49 × 10^3 joules, 3.9 seconds\",\n \"J\": \"0.00078 microfarad, 55 × 10^6 volts, 62 × 10^3 joules, 5.5 seconds\"\n }\n}
input
question.json
{
"question": "A spherical shell of paraffin is contained between two spherical conductors. The inner sphere has a radius of 0.60 meter, and the inner radius of the outer sphere is 0.70 meter. (a) What is the capacitance of the space between the conductors? (b) What is the largest electric potential difference which may be applied to the system if dielectric breakdown is to be avoided? (c) If the electric potential difference of (b) could be established between the two spheres, how much energy would be stored in the electric field? (d) If the heat loss from a particular building under certain conditions is 40,000 Btu per hour, for how long would the energy in the electric field of (c) maintain the temperature in this building if this energy were released?",
"options": {
"A": "0.00085 microfarad, 60 × 10^6 volts, 68 × 10^3 joules, 6.0 seconds",
"B": "0.00125 microfarad, 42 × 10^6 volts, 52 × 10^3 joules, 4.3 seconds",
"C": "0.00108 microfarad, 48 × 10^6 volts, 54 × 10^3 joules, 4.6 seconds",
"D": "0.00110 microfarad, 45 × 10^6 volts, 58 × 10^3 joules, 4.5 seconds",
"E": "0.00103 microfarad, 44 × 10^6 volts, 56 × 10^3 joules, 4.8 seconds",
"F": "0.00150 microfarad, 35 × 10^6 volts, 45 × 10^3 joules, 3.6 seconds",
"G": "0.00120 microfarad, 40 × 10^6 volts, 50 × 10^3 joules, 4.0 seconds",
"H": "0.00092 microfarad, 50 × 10^6 volts, 60 × 10^3 joules, 5.2 seconds",
"I": "0.00140 microfarad, 38 × 10^6 volts, 49 × 10^3 joules, 3.9 seconds",
"J": "0.00078 microfarad, 55 × 10^6 volts, 62 × 10^3 joules, 5.5 seconds"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_115/ files.
Scored by a judge this pinned commit does not carry.
▸case_116Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"For a short dipole antenna, evaluate the maximum effective apertureA_em.\",\n \"options\": {\n \"A\": \"0.119 wavelength\",\n \"B\": \"0.130 wavelength\",\n \"C\": \"0.112 wavelength^2\",\n \"D\": \"0.100 wavelength\",\n \"E\": \"0.123 wavelength^2\",\n \"F\": \"0.119 wavelength^2\",\n \"G\": \"0.150 wavelength^2\",\n \"H\": \"0.110 wavelength\",\n \"I\": \"0.135 wavelength^2\",\n \"J\": \"0.129 wavelength^2\"\n }\n}
input
question.json
{
"question": "For a short dipole antenna, evaluate the maximum effective apertureA_em.",
"options": {
"A": "0.119 wavelength",
"B": "0.130 wavelength",
"C": "0.112 wavelength^2",
"D": "0.100 wavelength",
"E": "0.123 wavelength^2",
"F": "0.119 wavelength^2",
"G": "0.150 wavelength^2",
"H": "0.110 wavelength",
"I": "0.135 wavelength^2",
"J": "0.129 wavelength^2"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_116/ files.
Scored by a judge this pinned commit does not carry.
▸case_117Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Fifteen years ago, two men who were fishing buddies moved onto vacant rural land owned by a woman they didn't know and built a small fishing shack on it. Twelve years ago, the men replaced the shack with a fish processing plant and a commercial fishing boat dock. The men maintained their commercial fishery operation on the land until one of them died intestate last year, leaving a sole heir. The period of time to acquire title by adverse possession in the jurisdiction is 10 years. The woman has now become aware of the changes that have occurred on the land. In an action to determine title, for whom should the court decide?\",\n \"options\": {\n \"A\": \"For the man who is still alive, because he is the surviving adverse possessor.\",\n \"B\": \"For the deceased man's heir, because he is the direct successor of the original adverse possessor.\",\n \"C\": \"For the man who is still alive and the deceased man's heir, because the men acquired title as tenants in common.\",\n \"D\": \"For the woman, because she was unaware of the changes that occurred on the land.\",\n \"E\": \"For the woman, because the use was changed by the men while they were in possession.\",\n \"F\": \"For the woman, because title cannot be claimed by two adverse possessors simultaneously.\",\n \"G\": \"For the man who is still alive and the deceased man's heir, because they have been using the land for commercial purposes.\",\n \"H\": \"For the woman, because the men changed the use of the land without her permission.\",\n \"I\": \"For the woman, because the men did not have her consent to build on her land.\",\n \"J\": \"For the man who is still alive, because he has been in continuous possession of the land for more than 10 years.\"\n }\n}
input
question.json
{
"question": "Fifteen years ago, two men who were fishing buddies moved onto vacant rural land owned by a woman they didn't know and built a small fishing shack on it. Twelve years ago, the men replaced the shack with a fish processing plant and a commercial fishing boat dock. The men maintained their commercial fishery operation on the land until one of them died intestate last year, leaving a sole heir. The period of time to acquire title by adverse possession in the jurisdiction is 10 years. The woman has now become aware of the changes that have occurred on the land. In an action to determine title, for whom should the court decide?",
"options": {
"A": "For the man who is still alive, because he is the surviving adverse possessor.",
"B": "For the deceased man's heir, because he is the direct successor of the original adverse possessor.",
"C": "For the man who is still alive and the deceased man's heir, because the men acquired title as tenants in common.",
"D": "For the woman, because she was unaware of the changes that occurred on the land.",
"E": "For the woman, because the use was changed by the men while they were in possession.",
"F": "For the woman, because title cannot be claimed by two adverse possessors simultaneously.",
"G": "For the man who is still alive and the deceased man's heir, because they have been using the land for commercial purposes.",
"H": "For the woman, because the men changed the use of the land without her permission.",
"I": "For the woman, because the men did not have her consent to build on her land.",
"J": "For the man who is still alive, because he has been in continuous possession of the land for more than 10 years."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_117/ files.
Scored by a judge this pinned commit does not carry.
▸case_118Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Third-year student is studying contracts. He has learned that generally all contractual rights may be assigned, but there are some exceptions. Which of the following is not an exception?\",\n \"options\": {\n \"A\": \"An assignment that substantially changes obligor's duty or risk.\",\n \"B\": \"An assignment that changes the nature of the contract.\",\n \"C\": \"An assignment that continues the obligor's duty or risk.\",\n \"D\": \"An assignment that transfers rights to an incapable party.\",\n \"E\": \"An assignment that causes undue hardship or loss to the obligor.\",\n \"F\": \"An assignment that reduces the value of the contract to the obligor.\",\n \"G\": \"An assignment of future rights from future contracts.\",\n \"H\": \"An assignment that is made without the consent of the other party.\",\n \"I\": \"An assignment not allowed by law.\"\n }\n}
input
question.json
{
"question": "Third-year student is studying contracts. He has learned that generally all contractual rights may be assigned, but there are some exceptions. Which of the following is not an exception?",
"options": {
"A": "An assignment that substantially changes obligor's duty or risk.",
"B": "An assignment that changes the nature of the contract.",
"C": "An assignment that continues the obligor's duty or risk.",
"D": "An assignment that transfers rights to an incapable party.",
"E": "An assignment that causes undue hardship or loss to the obligor.",
"F": "An assignment that reduces the value of the contract to the obligor.",
"G": "An assignment of future rights from future contracts.",
"H": "An assignment that is made without the consent of the other party.",
"I": "An assignment not allowed by law."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_118/ files.
Scored by a judge this pinned commit does not carry.
▸case_119Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A cast-iron flanged coupling is used to join two steel shafts 2(1/2) in., in diameter, delivering a maximum torque of 60 ft.-lb. The two halves of the couplings are joined together getherby 4 bolts, placed equidistant on a bolt circle of 3 in. diameter. If the design stress is 10,000 psi in shear for the bolt material, evaluate the diameter of the bolts.\",\n \"options\": {\n \"A\": \"0.1280 in.\",\n \"B\": \"0.1260 in.\",\n \"C\": \"0.1220 in.\",\n \"D\": \"0.1255 in.\",\n \"E\": \"0.1215 in.\",\n \"F\": \"0.1236 in.\",\n \"G\": \"0.1250 in.\",\n \"H\": \"0.1240 in.\",\n \"I\": \"0.1200 in.\",\n \"J\": \"0.1230 in.\"\n }\n}
input
question.json
{
"question": "A cast-iron flanged coupling is used to join two steel shafts 2(1/2) in., in diameter, delivering a maximum torque of 60 ft.-lb. The two halves of the couplings are joined together getherby 4 bolts, placed equidistant on a bolt circle of 3 in. diameter. If the design stress is 10,000 psi in shear for the bolt material, evaluate the diameter of the bolts.",
"options": {
"A": "0.1280 in.",
"B": "0.1260 in.",
"C": "0.1220 in.",
"D": "0.1255 in.",
"E": "0.1215 in.",
"F": "0.1236 in.",
"G": "0.1250 in.",
"H": "0.1240 in.",
"I": "0.1200 in.",
"J": "0.1230 in."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_119/ files.
Scored by a judge this pinned commit does not carry.
▸case_120Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A homeowner entered into a written contract with a house painter to have his house painted for $10,000. The contract stated in part: \\\"It is agreed that the house painter will paint the house using off-white latex paint, and will finish the job within 30 days from the execution of this agreement. If the homeowner finds the completed job to be satisfactory, he will pay the house painter $10,000 within one week after the job is completed. \\\"Which of the following is the most accurate statement regarding the homeowner's contractual obligation to pay the house painter $10,000?\",\n \"options\": {\n \"A\": \"The house painter's performance under the contract would be an express condition subsequent to the homeowner's duty of payment of the $10,000.\",\n \"B\": \"The performances of the homeowner and the house painter were, in essence, concurrent conditions.\",\n \"C\": \"The performances of the homeowner and the house painter were, in essence, independent conditions.\",\n \"D\": \"Payment of the $10,000 by the homeowner would be a concurrent condition to the house painter's duty of performance.\",\n \"E\": \"The house painter's performance under the contract would be an express condition precedent to the homeowner's duty of payment of the $10,000.\",\n \"F\": \"Payment of the $10,000 by the homeowner would be an express condition subsequent to the house painter's duty of performance.\",\n \"G\": \"Payment of the $10,000 by the homeowner would be an express condition precedent to the house painter's duty of performance.\",\n \"H\": \"The homeowner's satisfaction with the job is an implied condition precedent to the house painter's duty of performance.\",\n \"I\": \"The homeowner's satisfaction with the job would be an express condition precedent to the house painter's duty of performance.\",\n \"J\": \"The homeowner's satisfaction with the job is an implied condition subsequent to the house painter's duty of performance.\"\n }\n}
input
question.json
{
"question": "A homeowner entered into a written contract with a house painter to have his house painted for $10,000. The contract stated in part: \"It is agreed that the house painter will paint the house using off-white latex paint, and will finish the job within 30 days from the execution of this agreement. If the homeowner finds the completed job to be satisfactory, he will pay the house painter $10,000 within one week after the job is completed. \"Which of the following is the most accurate statement regarding the homeowner's contractual obligation to pay the house painter $10,000?",
"options": {
"A": "The house painter's performance under the contract would be an express condition subsequent to the homeowner's duty of payment of the $10,000.",
"B": "The performances of the homeowner and the house painter were, in essence, concurrent conditions.",
"C": "The performances of the homeowner and the house painter were, in essence, independent conditions.",
"D": "Payment of the $10,000 by the homeowner would be a concurrent condition to the house painter's duty of performance.",
"E": "The house painter's performance under the contract would be an express condition precedent to the homeowner's duty of payment of the $10,000.",
"F": "Payment of the $10,000 by the homeowner would be an express condition subsequent to the house painter's duty of performance.",
"G": "Payment of the $10,000 by the homeowner would be an express condition precedent to the house painter's duty of performance.",
"H": "The homeowner's satisfaction with the job is an implied condition precedent to the house painter's duty of performance.",
"I": "The homeowner's satisfaction with the job would be an express condition precedent to the house painter's duty of performance.",
"J": "The homeowner's satisfaction with the job is an implied condition subsequent to the house painter's duty of performance."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_120/ files.
Scored by a judge this pinned commit does not carry.
▸case_121Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A 2,500-kw 600-volt 16-pole generator has a lap-wound armature with 2,360 conductors. If the pole faces cover 65 per cent of the entire circumference, calculate the number of pole-face conductors in each pole of a compensating winding.\",\n \"options\": {\n \"A\": \"20 conductors\",\n \"B\": \"8 conductors\",\n \"C\": \"3 conductors\",\n \"D\": \"5 conductors\",\n \"E\": \"9 conductors\",\n \"F\": \"10 conductors\",\n \"G\": \"4 conductors\",\n \"H\": \"6 conductors\",\n \"I\": \"12 conductors\",\n \"J\": \"14 conductors\"\n }\n}
input
question.json
{
"question": "A 2,500-kw 600-volt 16-pole generator has a lap-wound armature with 2,360 conductors. If the pole faces cover 65 per cent of the entire circumference, calculate the number of pole-face conductors in each pole of a compensating winding.",
"options": {
"A": "20 conductors",
"B": "8 conductors",
"C": "3 conductors",
"D": "5 conductors",
"E": "9 conductors",
"F": "10 conductors",
"G": "4 conductors",
"H": "6 conductors",
"I": "12 conductors",
"J": "14 conductors"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_121/ files.
Scored by a judge this pinned commit does not carry.
▸case_122Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A young lawyer worked late at her office one night. When she finally decided to go home, she headed to the elevator, got in, and pushed the button for the ground floor. The elevator went all the way to the ground floor, but then just stopped there. The doors did not open. The lawyer tried pushing each of the buttons in the elevator and tried to use the intercom in the elevator to call for help, but nothing on the elevator's control panel seemed to be functioning. The lawyer was afraid that she would be stuck in the elevator all night. She began pounding on the doors and yelling. She started to feel panicked, hyperventilated, lost consciousness, and fell to the floor of the elevator. As she fell, she struck her head on the elevator wall, causing a large bump and bruise on her head. She was unconscious in the elevator for about 30 minutes before a security guard tried to use the elevator and realized it wasn't working. The guard managed to force the doors open and found the lawyer. At that moment, the lawyer regained consciousness. An investigation revealed that the elevator stopped working because it had not been properly maintained. The owner of the building had canceled all service and maintenance work on the elevator in order to save money. If the lawyer sues the building owner for false imprisonment, she will most likely\",\n \"options\": {\n \"A\": \"prevail, because she was confined in the elevator for an unreasonably long period of time.\",\n \"B\": \"prevail, because she was injured during her confinement in the elevator.\",\n \"C\": \"not prevail, because the building owner did not intentionally cause the lawyer to be confined.\",\n \"D\": \"not prevail, because the lawyer was unconscious and unaware of what was happening for most of the confinement period.\"\n }\n}
input
question.json
{
"question": "A young lawyer worked late at her office one night. When she finally decided to go home, she headed to the elevator, got in, and pushed the button for the ground floor. The elevator went all the way to the ground floor, but then just stopped there. The doors did not open. The lawyer tried pushing each of the buttons in the elevator and tried to use the intercom in the elevator to call for help, but nothing on the elevator's control panel seemed to be functioning. The lawyer was afraid that she would be stuck in the elevator all night. She began pounding on the doors and yelling. She started to feel panicked, hyperventilated, lost consciousness, and fell to the floor of the elevator. As she fell, she struck her head on the elevator wall, causing a large bump and bruise on her head. She was unconscious in the elevator for about 30 minutes before a security guard tried to use the elevator and realized it wasn't working. The guard managed to force the doors open and found the lawyer. At that moment, the lawyer regained consciousness. An investigation revealed that the elevator stopped working because it had not been properly maintained. The owner of the building had canceled all service and maintenance work on the elevator in order to save money. If the lawyer sues the building owner for false imprisonment, she will most likely",
"options": {
"A": "prevail, because she was confined in the elevator for an unreasonably long period of time.",
"B": "prevail, because she was injured during her confinement in the elevator.",
"C": "not prevail, because the building owner did not intentionally cause the lawyer to be confined.",
"D": "not prevail, because the lawyer was unconscious and unaware of what was happening for most of the confinement period."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_122/ files.
Scored by a judge this pinned commit does not carry.
▸case_123Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A man and a woman agreed that the woman would rob a bank and that the man would steal a car beforehand for the woman to use as a getaway vehicle. The man stole a car and parked it two blocks from the bank. He left the car key under the floor mat with a note saying that he wanted nothing more to do with the scheme. The next day, the woman robbed the bank and ran to the spot where the man had said he would leave the stolen car. She then escaped in the car and disappeared. She never shared any of the money with the man. In a jurisdiction that has adopted the bilateral requirement for conspiracy, can the man properly be convicted of conspiring with the woman to rob the bank?\",\n \"options\": {\n \"A\": \"Yes, because the man was a part of the initial planning of the robbery.\",\n \"B\": \"Yes, because the man committed a crime (stealing a car) in furtherance of the conspiracy.\",\n \"C\": \"Yes, because the robbery was successful due in part to the man's actions.\",\n \"D\": \"Yes, because the man provided the getaway vehicle.\",\n \"E\": \"Yes, because there was an agreement to rob the bank and an overt act in furtherance of the agreement.\",\n \"F\": \"No, because the man expressed his intention to withdraw from the conspiracy.\",\n \"G\": \"No, because the man received no benefit from the robbery.\",\n \"H\": \"No, because the woman did not share the money with the man.\",\n \"I\": \"No, because the man did not participate in the actual robbery.\"\n }\n}
input
question.json
{
"question": "A man and a woman agreed that the woman would rob a bank and that the man would steal a car beforehand for the woman to use as a getaway vehicle. The man stole a car and parked it two blocks from the bank. He left the car key under the floor mat with a note saying that he wanted nothing more to do with the scheme. The next day, the woman robbed the bank and ran to the spot where the man had said he would leave the stolen car. She then escaped in the car and disappeared. She never shared any of the money with the man. In a jurisdiction that has adopted the bilateral requirement for conspiracy, can the man properly be convicted of conspiring with the woman to rob the bank?",
"options": {
"A": "Yes, because the man was a part of the initial planning of the robbery.",
"B": "Yes, because the man committed a crime (stealing a car) in furtherance of the conspiracy.",
"C": "Yes, because the robbery was successful due in part to the man's actions.",
"D": "Yes, because the man provided the getaway vehicle.",
"E": "Yes, because there was an agreement to rob the bank and an overt act in furtherance of the agreement.",
"F": "No, because the man expressed his intention to withdraw from the conspiracy.",
"G": "No, because the man received no benefit from the robbery.",
"H": "No, because the woman did not share the money with the man.",
"I": "No, because the man did not participate in the actual robbery."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_123/ files.
Scored by a judge this pinned commit does not carry.
▸case_124Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Suppose an antenna has a power input of 40\\\\piW and an efficiencyof 98 percent. If the radiation intensity has been foundto have a maximum value of 200 W/unit solid angle, findthe directivity and gain of the antenna.\",\n \"options\": {\n \"A\": \"Directivity: 30.41 or 20.10 dB, Gain: 30 or 20.01 dB\",\n \"B\": \"Directivity: 12.41 or 7.10 dB, Gain: 12 or 7.01 dB\",\n \"C\": \"Directivity: 25.41 or 15.10 dB, Gain: 25 or 15.01 dB\",\n \"D\": \"Directivity: 10.41 or 5.10 dB, Gain: 10 or 5.01 dB\",\n \"E\": \"Directivity: 40.41 or 30.10 dB, Gain: 40 or 30.01 dB\",\n \"F\": \"Directivity: 35.41 or 25.10 dB, Gain: 35 or 25.01 dB\",\n \"G\": \"Directivity: 18.41 or 11.10 dB, Gain: 18 or 11.01 dB\",\n \"H\": \"Directivity: 15.41 or 9.10 dB, Gain: 15 or 9.01 dB\",\n \"I\": \"Directivity: 22.41 or 14.10 dB, Gain: 22 or 14.01 dB\",\n \"J\": \"Directivity: 20.41 or 13.10 dB, Gain: 20 or 13.01 dB\"\n }\n}
input
question.json
{
"question": "Suppose an antenna has a power input of 40\\piW and an efficiencyof 98 percent. If the radiation intensity has been foundto have a maximum value of 200 W/unit solid angle, findthe directivity and gain of the antenna.",
"options": {
"A": "Directivity: 30.41 or 20.10 dB, Gain: 30 or 20.01 dB",
"B": "Directivity: 12.41 or 7.10 dB, Gain: 12 or 7.01 dB",
"C": "Directivity: 25.41 or 15.10 dB, Gain: 25 or 15.01 dB",
"D": "Directivity: 10.41 or 5.10 dB, Gain: 10 or 5.01 dB",
"E": "Directivity: 40.41 or 30.10 dB, Gain: 40 or 30.01 dB",
"F": "Directivity: 35.41 or 25.10 dB, Gain: 35 or 25.01 dB",
"G": "Directivity: 18.41 or 11.10 dB, Gain: 18 or 11.01 dB",
"H": "Directivity: 15.41 or 9.10 dB, Gain: 15 or 9.01 dB",
"I": "Directivity: 22.41 or 14.10 dB, Gain: 22 or 14.01 dB",
"J": "Directivity: 20.41 or 13.10 dB, Gain: 20 or 13.01 dB"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_124/ files.
Scored by a judge this pinned commit does not carry.
▸case_125Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A man drinks a great deal of punch at a party. He was unaware that it contained PCP. Later in the evening, in an uncontrollable psychotic rage brought on by the drugs he kills someone at the party. The prosecution asks for second degree murder. What defense would have the best chance to succeed under the circumstances?\",\n \"options\": {\n \"A\": \"Self-Defense\",\n \"B\": \"Duress\",\n \"C\": \"Involuntary intoxication\",\n \"D\": \"Necessity\",\n \"E\": \"Irresistible Impulse\",\n \"F\": \"Heat of Passion\",\n \"G\": \"Affluenza\",\n \"H\": \"Voluntary intoxication\",\n \"I\": \"Ignorance or Mistake\",\n \"J\": \"Insanity\"\n }\n}
input
question.json
{
"question": "A man drinks a great deal of punch at a party. He was unaware that it contained PCP. Later in the evening, in an uncontrollable psychotic rage brought on by the drugs he kills someone at the party. The prosecution asks for second degree murder. What defense would have the best chance to succeed under the circumstances?",
"options": {
"A": "Self-Defense",
"B": "Duress",
"C": "Involuntary intoxication",
"D": "Necessity",
"E": "Irresistible Impulse",
"F": "Heat of Passion",
"G": "Affluenza",
"H": "Voluntary intoxication",
"I": "Ignorance or Mistake",
"J": "Insanity"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_125/ files.
Scored by a judge this pinned commit does not carry.
▸case_126Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A builder sold a new house to a buyer for use as the buyer's residence. The buyer paid 10% of the purchase price and financed the rest by executing a promissory note and purchase money mortgage to the builder. A year later, the buyer missed several mortgage payments to the builder and became unable to make payments to the builder and became unable to make payments. During that year, property values in the neighborhood declined substantially. The builder suggested that the buyer deed the house back to the builder to settle all claims and avoid the costs and other disadvantages of foreclosure. The buyer deeded the house back to the builder. Does the builder now own fee simple title to the house?\",\n \"options\": {\n \"A\": \"No, because the builder cannot accept the deed while also holding the mortgage.\",\n \"B\": \"No, because the owner of a personal residence cannot waive the right to foreclosure.\",\n \"C\": \"Yes, because the transaction was reasonable and fair under the circumstances.\",\n \"D\": \"No, because the builder coerced the buyer into deeding the house back.\",\n \"E\": \"Yes, because the buyer defaulted on the mortgage payments.\",\n \"F\": \"Yes, because the builder originally held the mortgage.\",\n \"G\": \"Yes, because of the doctrine of equitable redemption.\",\n \"H\": \"No, because the buyer still technically owns the house until foreclosure.\",\n \"I\": \"No, because the deed back to the builder constitutes a disguised mortgage.\"\n }\n}
input
question.json
{
"question": "A builder sold a new house to a buyer for use as the buyer's residence. The buyer paid 10% of the purchase price and financed the rest by executing a promissory note and purchase money mortgage to the builder. A year later, the buyer missed several mortgage payments to the builder and became unable to make payments to the builder and became unable to make payments. During that year, property values in the neighborhood declined substantially. The builder suggested that the buyer deed the house back to the builder to settle all claims and avoid the costs and other disadvantages of foreclosure. The buyer deeded the house back to the builder. Does the builder now own fee simple title to the house?",
"options": {
"A": "No, because the builder cannot accept the deed while also holding the mortgage.",
"B": "No, because the owner of a personal residence cannot waive the right to foreclosure.",
"C": "Yes, because the transaction was reasonable and fair under the circumstances.",
"D": "No, because the builder coerced the buyer into deeding the house back.",
"E": "Yes, because the buyer defaulted on the mortgage payments.",
"F": "Yes, because the builder originally held the mortgage.",
"G": "Yes, because of the doctrine of equitable redemption.",
"H": "No, because the buyer still technically owns the house until foreclosure.",
"I": "No, because the deed back to the builder constitutes a disguised mortgage."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_126/ files.
Scored by a judge this pinned commit does not carry.
▸case_127Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"In a jurisdiction using the grantor \u0080\u0094grantee indices, the following events have occurred in the order listed below:(1) In 1993, a borrower mortgaged a ranch to a mortgage company to secure a note for $25,000, due on December 31, 1993, with 9V2 percent interest per annum. (2) In 1996, the mortgage company recorded the mortgage on the ranch. (3) In 2007, in a signed writing, which the borrower delivered to his daughter, the borrower promised to convey the ranch to the daughter by a quitclaim deed and pay the mortgage debt when it came due. (4) In 2008, the borrower delivered to the daughter a quitclaim deed that made no reference to the mortgage. If the borrower thereafter defaulted on the mortgage and the mortgage company brought an in personam action against the daughter to recover the amount due on the mortgage debt, the mortgagee will probably\",\n \"options\": {\n \"A\": \"not succeed, because the daughter was not a party to the original mortgage contract.\",\n \"B\": \"succeed, because the mortgage company has the right to seek payment from any party associated with the property.\",\n \"C\": \"succeed, because there was an implied delegation of duties to the daughter.\",\n \"D\": \"succeed, because the mortgage company is a third-party beneficiary of the agreement between the borrower and the daughter.\",\n \"E\": \"not succeed, because the daughter did not promise to pay the mortgage debt.\",\n \"F\": \"succeed, because the daughter took ownership of the property with the quitclaim deed.\",\n \"G\": \"not succeed, because the mortgage company did not record the mortgage until 1996.\",\n \"H\": \"not succeed, because the daughter was only promised the property, not the debt.\",\n \"I\": \"succeed, because the borrower defaulted on the mortgage.\",\n \"J\": \"not succeed, because the law does not permit the mortgagor to delegate duties under this type of a property transfer.\"\n }\n}
input
question.json
{
"question": "In a jurisdiction using the grantor grantee indices, the following events have occurred in the order listed below:(1) In 1993, a borrower mortgaged a ranch to a mortgage company to secure a note for $25,000, due on December 31, 1993, with 9V2 percent interest per annum. (2) In 1996, the mortgage company recorded the mortgage on the ranch. (3) In 2007, in a signed writing, which the borrower delivered to his daughter, the borrower promised to convey the ranch to the daughter by a quitclaim deed and pay the mortgage debt when it came due. (4) In 2008, the borrower delivered to the daughter a quitclaim deed that made no reference to the mortgage. If the borrower thereafter defaulted on the mortgage and the mortgage company brought an in personam action against the daughter to recover the amount due on the mortgage debt, the mortgagee will probably",
"options": {
"A": "not succeed, because the daughter was not a party to the original mortgage contract.",
"B": "succeed, because the mortgage company has the right to seek payment from any party associated with the property.",
"C": "succeed, because there was an implied delegation of duties to the daughter.",
"D": "succeed, because the mortgage company is a third-party beneficiary of the agreement between the borrower and the daughter.",
"E": "not succeed, because the daughter did not promise to pay the mortgage debt.",
"F": "succeed, because the daughter took ownership of the property with the quitclaim deed.",
"G": "not succeed, because the mortgage company did not record the mortgage until 1996.",
"H": "not succeed, because the daughter was only promised the property, not the debt.",
"I": "succeed, because the borrower defaulted on the mortgage.",
"J": "not succeed, because the law does not permit the mortgagor to delegate duties under this type of a property transfer."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_127/ files.
Scored by a judge this pinned commit does not carry.
▸case_128Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Determine the inverse Laplace transform of F(s), where F(s) is given by F(s) = [(3s + 1) / {(s - 1)(s^2 + 1)}]\",\n \"options\": {\n \"A\": \"3e^t + sin t - cos t\",\n \"B\": \"e^t - cos t + sin t\",\n \"C\": \"e^t - sin t - cos t\",\n \"D\": \"e^t + cos t - 2 sin t\",\n \"E\": \"3e^t - cos t - 2 sin t\",\n \"F\": \"2e^t + sin t + 2 cos t\",\n \"G\": \"2e^t - 2 cos t + sin t\",\n \"H\": \"2e^t - 2 sin t + cos t\",\n \"I\": \"e^t - 2 cos t + 2 sin t\",\n \"J\": \"2e^t + 2 cos t - sin t\"\n }\n}
input
question.json
{
"question": "Determine the inverse Laplace transform of F(s), where F(s) is given by F(s) = [(3s + 1) / {(s - 1)(s^2 + 1)}]",
"options": {
"A": "3e^t + sin t - cos t",
"B": "e^t - cos t + sin t",
"C": "e^t - sin t - cos t",
"D": "e^t + cos t - 2 sin t",
"E": "3e^t - cos t - 2 sin t",
"F": "2e^t + sin t + 2 cos t",
"G": "2e^t - 2 cos t + sin t",
"H": "2e^t - 2 sin t + cos t",
"I": "e^t - 2 cos t + 2 sin t",
"J": "2e^t + 2 cos t - sin t"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_128/ files.
Scored by a judge this pinned commit does not carry.
▸case_129Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"The function f is given by f(x, y, z) = [z / (x^2 + y^2)](1) Compute \\\\nablaf in cylindricalcoordinates\\\\rho, \\\\texttheta, z.\",\n \"options\": {\n \"A\": \"- (2z / \\nho^2)u_\\nho+ (1 / \\nho^3)u_z\",\n \"B\": \"\\\\( (z / \\\\rho^3)u_\\\\rho + (1 / \\\\rho)u_\\\\theta - (1 / \\\\rho^3)u_z \\\\)\",\n \"C\": \"- (2z / \\nho^3)u_\\nho+ (1 / \\nho)u_z\",\n \"D\": \"\\\\( - (z / \\\\rho^2)u_\\\\rho + (1 / \\\\rho^3)u_z \\\\)\",\n \"E\": \"\\\\( - (1 / \\\\rho^2)u_\\\\rho - (2z / \\\\rho^3)u_\\\\theta + (2 / \\\\rho)u_z \\\\)\",\n \"F\": \"\\\\( (1 / \\\\rho^2)u_\\\\rho + (z / \\\\rho^3)u_\\\\theta + (2 / \\\\rho^3)u_z \\\\)\",\n \"G\": \"- (2z / \\nho^3)u_\\nho+ (1 / \\nho^2)u_z\",\n \"H\": \"\\\\( (2z / \\\\rho^3)u_\\\\rho - (1 / \\\\rho)u_\\\\theta + (1 / \\\\rho^2)u_z \\\\)\",\n \"I\": \"(2z / \\nho^3)u_\\nho+ (1 / \\nho^2)u_z\",\n \"J\": \"\\\\( (2 / \\\\rho^3)u_\\\\rho - (z / \\\\rho^2)u_\\\\theta - (1 / \\\\rho)u_z \\\\)\"\n }\n}
input
question.json
{
"question": "The function f is given by f(x, y, z) = [z / (x^2 + y^2)](1) Compute \\nablaf in cylindricalcoordinates\\rho, \\texttheta, z.",
"options": {
"A": "- (2z / \nho^2)u_\nho+ (1 / \nho^3)u_z",
"B": "\\( (z / \\rho^3)u_\\rho + (1 / \\rho)u_\\theta - (1 / \\rho^3)u_z \\)",
"C": "- (2z / \nho^3)u_\nho+ (1 / \nho)u_z",
"D": "\\( - (z / \\rho^2)u_\\rho + (1 / \\rho^3)u_z \\)",
"E": "\\( - (1 / \\rho^2)u_\\rho - (2z / \\rho^3)u_\\theta + (2 / \\rho)u_z \\)",
"F": "\\( (1 / \\rho^2)u_\\rho + (z / \\rho^3)u_\\theta + (2 / \\rho^3)u_z \\)",
"G": "- (2z / \nho^3)u_\nho+ (1 / \nho^2)u_z",
"H": "\\( (2z / \\rho^3)u_\\rho - (1 / \\rho)u_\\theta + (1 / \\rho^2)u_z \\)",
"I": "(2z / \nho^3)u_\nho+ (1 / \nho^2)u_z",
"J": "\\( (2 / \\rho^3)u_\\rho - (z / \\rho^2)u_\\theta - (1 / \\rho)u_z \\)"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_129/ files.
Scored by a judge this pinned commit does not carry.
▸case_130Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Two infinite parallel metal plates are charged with equal surface charge density of the same polarity. The electric field in the gap between the plates is\",\n \"options\": {\n \"A\": \"inversely proportional to the distance between the plates.\",\n \"B\": \"quadruple the field produced by one plate.\",\n \"C\": \"zero.\",\n \"D\": \"dependent on the thickness of the plates.\",\n \"E\": \"dependent on coordinates of field points.\",\n \"F\": \"proportional to the square of the distance between the plates.\",\n \"G\": \"double the field produced by one plate.\",\n \"H\": \"half the field produced by one plate.\",\n \"I\": \"triple the field produced by one plate.\",\n \"J\": \"same as that produced by one plate.\"\n }\n}
input
question.json
{
"question": "Two infinite parallel metal plates are charged with equal surface charge density of the same polarity. The electric field in the gap between the plates is",
"options": {
"A": "inversely proportional to the distance between the plates.",
"B": "quadruple the field produced by one plate.",
"C": "zero.",
"D": "dependent on the thickness of the plates.",
"E": "dependent on coordinates of field points.",
"F": "proportional to the square of the distance between the plates.",
"G": "double the field produced by one plate.",
"H": "half the field produced by one plate.",
"I": "triple the field produced by one plate.",
"J": "same as that produced by one plate."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_130/ files.
Scored by a judge this pinned commit does not carry.
▸case_131Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A businessman was the owner of an idyllic lake and wooded area. In 1980, the businessman conveyed the land to the state \\\"on the condition that\\\" it be used as recreational property only. Two years later, the state decided that they wanted a small industrial park attached to the corner of the land. The state went ahead and developed a small portion of the land as industrial space, making it part of a larger industrial park that was adjacent to the land. The businessman notified the state that he was the owner of the property because the state violated the conditions of the conveyance, and that he was taking possession of the property back. What was the interest that was created by the conveyance and can the businessman get the property back?\",\n \"options\": {\n \"A\": \"The interest created is an indefeasible estate, and the owner cannot do anything to enforce a condition that is against public policy and never enforceable in the first place.\",\n \"B\": \"The interest created is a life estate, and the property will revert back to the businessman only after the life tenant's death.\",\n \"C\": \"The interest created is a reverter, and the property instantly reverts back to the grantor upon the occurrence of the specified violation.\",\n \"D\": \"The interest created is a fee simple subject to a condition subsequent, and the original owner can get the property back only by taking affirmative steps to exercise his right of entry.\",\n \"E\": \"The interest created is a leasehold estate, and the businessman can reclaim the property only after the lease period ends.\",\n \"F\": \"The interest is a remainder, and the property will only revert back to the original owner after the state's interest ends.\",\n \"G\": \"The interest created is an easement, and the businessman has no right to reclaim the property but can prevent the state from using it for industrial purposes.\",\n \"H\": \"The interest is a fee simple subject to an executory limitation, and if the condition occurs the property reverts in fee simple to a pre-designated third party.\",\n \"I\": \"The interest created is a fee simple absolute, and the owner cannot reclaim the property as it now completely belongs to the state.\",\n \"J\": \"The interest is a fee simple determinable, and the property instantly reverts back to the businessman upon violation of the condition.\"\n }\n}
input
question.json
{
"question": "A businessman was the owner of an idyllic lake and wooded area. In 1980, the businessman conveyed the land to the state \"on the condition that\" it be used as recreational property only. Two years later, the state decided that they wanted a small industrial park attached to the corner of the land. The state went ahead and developed a small portion of the land as industrial space, making it part of a larger industrial park that was adjacent to the land. The businessman notified the state that he was the owner of the property because the state violated the conditions of the conveyance, and that he was taking possession of the property back. What was the interest that was created by the conveyance and can the businessman get the property back?",
"options": {
"A": "The interest created is an indefeasible estate, and the owner cannot do anything to enforce a condition that is against public policy and never enforceable in the first place.",
"B": "The interest created is a life estate, and the property will revert back to the businessman only after the life tenant's death.",
"C": "The interest created is a reverter, and the property instantly reverts back to the grantor upon the occurrence of the specified violation.",
"D": "The interest created is a fee simple subject to a condition subsequent, and the original owner can get the property back only by taking affirmative steps to exercise his right of entry.",
"E": "The interest created is a leasehold estate, and the businessman can reclaim the property only after the lease period ends.",
"F": "The interest is a remainder, and the property will only revert back to the original owner after the state's interest ends.",
"G": "The interest created is an easement, and the businessman has no right to reclaim the property but can prevent the state from using it for industrial purposes.",
"H": "The interest is a fee simple subject to an executory limitation, and if the condition occurs the property reverts in fee simple to a pre-designated third party.",
"I": "The interest created is a fee simple absolute, and the owner cannot reclaim the property as it now completely belongs to the state.",
"J": "The interest is a fee simple determinable, and the property instantly reverts back to the businessman upon violation of the condition."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_131/ files.
Scored by a judge this pinned commit does not carry.
▸case_132Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Under common law principles, which of the following is not correct regarding the duty owed by a possessor of land towards an undiscovered trespasser?\",\n \"options\": {\n \"A\": \"No duty for artificial conditions.\",\n \"B\": \"A duty for natural conditions.\",\n \"C\": \"No duty for natural conditions.\",\n \"D\": \"No duty for active operations.\"\n }\n}
input
question.json
{
"question": "Under common law principles, which of the following is not correct regarding the duty owed by a possessor of land towards an undiscovered trespasser?",
"options": {
"A": "No duty for artificial conditions.",
"B": "A duty for natural conditions.",
"C": "No duty for natural conditions.",
"D": "No duty for active operations."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_132/ files.
Scored by a judge this pinned commit does not carry.
▸case_133Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A homeowner buys a new leaf blower and is so impressed with its power that he gets carried away and blows large amounts of his work onto the next door neighbor's yard. In addition to leaves, he also blew large amounts of pest-infested plant debris onto the neighbor's property. The material has accumulated into large, unsightly piles and caused damage to the neighbor's healthy plants. He has to pay workers to haul the material away and he incurs expenses to replace infected perennials. The neighbor sues for the tort of trespass. The homeowner files a motion to dismiss because the neighbor cannot prove that homeowner acted with an intent to trespass or to do harm. Will the court grant the motion to dismiss?\",\n \"options\": {\n \"A\": \"Yes, because the homeowner expected the wind to carry the debris away and did not think that it would accumulate on the neighbor's property.\",\n \"B\": \"No, because the homeowner's actions resulted in a direct invasion of the neighbor's property.\",\n \"C\": \"Yes, because the debris was not intentionally directed at the neighbor's property.\",\n \"D\": \"Yes, because the homeowner was not aware of the pest-infested debris.\",\n \"E\": \"No, because the neighbor was able to prove financial loss from the damage caused.\",\n \"F\": \"No, because trespass is based on an intentional act of entering the land or sending something onto the land, and the actor does not have to intend harm to be liable for the tort.\",\n \"G\": \"Yes, because the homeowner had no practical way of controlling where the material went and he acted without malice or ill will.\",\n \"H\": \"Yes, because the homeowner did not physically enter the neighbor's property.\",\n \"I\": \"No, because the infected debris constituted an abnormally dangerous condition and homeowner could be held strictly liable for the trespass.\",\n \"J\": \"No, because the act of blowing debris onto the neighbor's land constitutes a trespass regardless of intent.\"\n }\n}
input
question.json
{
"question": "A homeowner buys a new leaf blower and is so impressed with its power that he gets carried away and blows large amounts of his work onto the next door neighbor's yard. In addition to leaves, he also blew large amounts of pest-infested plant debris onto the neighbor's property. The material has accumulated into large, unsightly piles and caused damage to the neighbor's healthy plants. He has to pay workers to haul the material away and he incurs expenses to replace infected perennials. The neighbor sues for the tort of trespass. The homeowner files a motion to dismiss because the neighbor cannot prove that homeowner acted with an intent to trespass or to do harm. Will the court grant the motion to dismiss?",
"options": {
"A": "Yes, because the homeowner expected the wind to carry the debris away and did not think that it would accumulate on the neighbor's property.",
"B": "No, because the homeowner's actions resulted in a direct invasion of the neighbor's property.",
"C": "Yes, because the debris was not intentionally directed at the neighbor's property.",
"D": "Yes, because the homeowner was not aware of the pest-infested debris.",
"E": "No, because the neighbor was able to prove financial loss from the damage caused.",
"F": "No, because trespass is based on an intentional act of entering the land or sending something onto the land, and the actor does not have to intend harm to be liable for the tort.",
"G": "Yes, because the homeowner had no practical way of controlling where the material went and he acted without malice or ill will.",
"H": "Yes, because the homeowner did not physically enter the neighbor's property.",
"I": "No, because the infected debris constituted an abnormally dangerous condition and homeowner could be held strictly liable for the trespass.",
"J": "No, because the act of blowing debris onto the neighbor's land constitutes a trespass regardless of intent."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_133/ files.
Scored by a judge this pinned commit does not carry.
▸case_134Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Calculate the loss of head in a pipe (1/4) in. in diameter and 16 ft. long, when water flows at half the critical velocity. The critical velocity occurs when Reynolds number is 2500. Take viscosity \\\\mu = 1.01 centipoise = 0.679 × 10^-3 lbm/ft-sec.\",\n \"options\": {\n \"A\": \"3.12 in\",\n \"B\": \"7.5 in\",\n \"C\": \"0.8 in\",\n \"D\": \"6.3 in\",\n \"E\": \"2.5 in\",\n \"F\": \"5.0 in\",\n \"G\": \"1.75 in\",\n \"H\": \"4.2 in\",\n \"I\": \"4.8 in\",\n \"J\": \"2.0 in\"\n }\n}
input
question.json
{
"question": "Calculate the loss of head in a pipe (1/4) in. in diameter and 16 ft. long, when water flows at half the critical velocity. The critical velocity occurs when Reynolds number is 2500. Take viscosity \\mu = 1.01 centipoise = 0.679 × 10^-3 lbm/ft-sec.",
"options": {
"A": "3.12 in",
"B": "7.5 in",
"C": "0.8 in",
"D": "6.3 in",
"E": "2.5 in",
"F": "5.0 in",
"G": "1.75 in",
"H": "4.2 in",
"I": "4.8 in",
"J": "2.0 in"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_134/ files.
Scored by a judge this pinned commit does not carry.
▸case_135Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A 3mm thick aluminium alloy plate at 200°C is suddenly quenchedinto water at 10°C. Determine the time required forthe plate to reach the temperature of 30°C. The following datamay be used. Density of aluminium \\\\rho = 3000kg/m^3 Specific heat of aluminium C_p = 0.2kcal/kg°C Plate dimensions = 20 cm × 20 cm Heat transfer coefficient h = 5000 kcal/m.^2hr. °C\",\n \"options\": {\n \"A\": \"4.0 seconds\",\n \"B\": \"2.8 seconds\",\n \"C\": \"6.3 seconds\",\n \"D\": \"5.5 seconds\",\n \"E\": \"3.2 seconds\",\n \"F\": \"1.45 seconds\",\n \"G\": \"2.5 seconds\",\n \"H\": \"3.6 seconds\",\n \"I\": \"0.98 seconds\",\n \"J\": \"0.75 seconds\"\n }\n}
input
question.json
{
"question": "A 3mm thick aluminium alloy plate at 200°C is suddenly quenchedinto water at 10°C. Determine the time required forthe plate to reach the temperature of 30°C. The following datamay be used. Density of aluminium \\rho = 3000kg/m^3 Specific heat of aluminium C_p = 0.2kcal/kg°C Plate dimensions = 20 cm × 20 cm Heat transfer coefficient h = 5000 kcal/m.^2hr. °C",
"options": {
"A": "4.0 seconds",
"B": "2.8 seconds",
"C": "6.3 seconds",
"D": "5.5 seconds",
"E": "3.2 seconds",
"F": "1.45 seconds",
"G": "2.5 seconds",
"H": "3.6 seconds",
"I": "0.98 seconds",
"J": "0.75 seconds"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_135/ files.
Scored by a judge this pinned commit does not carry.
▸case_136Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Two large tanks, each contain a mixture of nitrogen (A) and oxygen(B), but at different concentrations. Tank 1 has 85 molepercent N_2 and 15 mole percent O_2, while tank 2 has 25 mole percent N_2 and 75 mole percent O_2. A tube 1.5 m long with an inside diameter of 0.150 m connects the two tanks. If the pressure is 1atmand the temperature is 0°C, calculate the rate of diffusion of N_2 between the tanks assuming steady state transfer.\",\n \"options\": {\n \"A\": \"7.85 × 10^-8 kgmol/sec\",\n \"B\": \"6.02 × 10^-9 kgmol/sec\",\n \"C\": \"2.0 × 10^-9kgmol/ sec\",\n \"D\": \"1.23 × 10^-5 kgmol/sec\",\n \"E\": \"5.71 × 10^-7kgmol/ sec\",\n \"F\": \"5.71 × 10^-9kgmol/ sec\",\n \"G\": \"4.22 × 10^-7 kgmol/sec\",\n \"H\": \"9.67 × 10^-10 kgmol/sec\",\n \"I\": \"3.14 × 10^-6 kgmol/sec\",\n \"J\": \"1.81 × 10^-5kgmol/ sec\"\n }\n}
input
question.json
{
"question": "Two large tanks, each contain a mixture of nitrogen (A) and oxygen(B), but at different concentrations. Tank 1 has 85 molepercent N_2 and 15 mole percent O_2, while tank 2 has 25 mole percent N_2 and 75 mole percent O_2. A tube 1.5 m long with an inside diameter of 0.150 m connects the two tanks. If the pressure is 1atmand the temperature is 0°C, calculate the rate of diffusion of N_2 between the tanks assuming steady state transfer.",
"options": {
"A": "7.85 × 10^-8 kgmol/sec",
"B": "6.02 × 10^-9 kgmol/sec",
"C": "2.0 × 10^-9kgmol/ sec",
"D": "1.23 × 10^-5 kgmol/sec",
"E": "5.71 × 10^-7kgmol/ sec",
"F": "5.71 × 10^-9kgmol/ sec",
"G": "4.22 × 10^-7 kgmol/sec",
"H": "9.67 × 10^-10 kgmol/sec",
"I": "3.14 × 10^-6 kgmol/sec",
"J": "1.81 × 10^-5kgmol/ sec"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_136/ files.
Scored by a judge this pinned commit does not carry.
▸case_137Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"The current through an inductor with inductance L = 10^-3 henry is given as i_L (t) = 0.1 sin 10^6t. Find the voltage\\\\nu_L(t) across this inductor.\",\n \"options\": {\n \"A\": \"1000sin10^6t\",\n \"B\": \"100sin10^6t\",\n \"C\": \"50cos10^6t\",\n \"D\": \"1000cos10^6t\",\n \"E\": \"0.1cos10^6t\",\n \"F\": \"0.1sin10^6t\",\n \"G\": \"100cos10^6t\",\n \"H\": \"10sin10^6t\",\n \"I\": \"10^-3cos10^6t\",\n \"J\": \"10cos10^6t\"\n }\n}
input
question.json
{
"question": "The current through an inductor with inductance L = 10^-3 henry is given as i_L (t) = 0.1 sin 10^6t. Find the voltage\\nu_L(t) across this inductor.",
"options": {
"A": "1000sin10^6t",
"B": "100sin10^6t",
"C": "50cos10^6t",
"D": "1000cos10^6t",
"E": "0.1cos10^6t",
"F": "0.1sin10^6t",
"G": "100cos10^6t",
"H": "10sin10^6t",
"I": "10^-3cos10^6t",
"J": "10cos10^6t"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_137/ files.
Scored by a judge this pinned commit does not carry.
▸case_138Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A 4 in. schedule 40 wrought iron pipe is covered with 2 in. thick layer of magnesia insulation (k = 0.041 Btu/hr-ft-°F) and 1(1/4) in. thickness of glass wool (k = 0.032 Btu/hr-ft-°F). If the outer surface of the pipe is at 700°F and the outer surface of the glass wool insulation is at 100°F, determine the rate of heat loss per foot length of the pipe.\",\n \"options\": {\n \"A\": \"175.9 Btu/(hr)(ft. of pipe)\",\n \"B\": \"190.3 Btu/(hr)(ft. of pipe)\",\n \"C\": \"160.2 Btu/(hr)(ft. of pipe)\",\n \"D\": \"220.8 Btu/(hr)(ft. of pipe)\",\n \"E\": \"150.1 Btu/(hr)(ft. of pipe)\",\n \"F\": \"130.7 Btu/(hr)(ft. of pipe)\",\n \"G\": \"120.5 Btu/(hr)(ft. of pipe)\",\n \"H\": \"180.3 Btu/(hr)(ft. of pipe)\",\n \"I\": \"140.4 Btu/(hr)(ft. of pipe)\",\n \"J\": \"200.6 Btu/(hr)(ft. of pipe)\"\n }\n}
input
question.json
{
"question": "A 4 in. schedule 40 wrought iron pipe is covered with 2 in. thick layer of magnesia insulation (k = 0.041 Btu/hr-ft-°F) and 1(1/4) in. thickness of glass wool (k = 0.032 Btu/hr-ft-°F). If the outer surface of the pipe is at 700°F and the outer surface of the glass wool insulation is at 100°F, determine the rate of heat loss per foot length of the pipe.",
"options": {
"A": "175.9 Btu/(hr)(ft. of pipe)",
"B": "190.3 Btu/(hr)(ft. of pipe)",
"C": "160.2 Btu/(hr)(ft. of pipe)",
"D": "220.8 Btu/(hr)(ft. of pipe)",
"E": "150.1 Btu/(hr)(ft. of pipe)",
"F": "130.7 Btu/(hr)(ft. of pipe)",
"G": "120.5 Btu/(hr)(ft. of pipe)",
"H": "180.3 Btu/(hr)(ft. of pipe)",
"I": "140.4 Btu/(hr)(ft. of pipe)",
"J": "200.6 Btu/(hr)(ft. of pipe)"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_138/ files.
Scored by a judge this pinned commit does not carry.
▸case_139Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"In 1888, a landowner owned a dairy farm. The landowner conveyed this real property to his son in1938. In 1953, the son conveyed the dairy farm to his friend. This deed was not recorded until after the son's death in 1957. In 1956, the son mortgaged the dairy farm to the bank. The mortgage instrument, which was recorded in 1956, recited that it was subordinate to a mortgage on the same land given by the son to an investor in 1936 and recorded in 1936. In that instrument the son purported to grant the investor a mortgage on the dairy farm. In 1979, the friend conveyed the dairy farm to a farmer. This deed was duly recorded, but did not mention any mortgage. In 2008, a buyer entered into an agreement with the farmer, whereby the farmer would convey the dairy farm in fee simple to the buyer for the sum of $75,000. The closing date was set for January 15, 2009. All of the deeds mentioned in the aforementioned transactions are general warranty deeds. In addition, this jurisdiction has a notice-type recording statute and follows a title theory for mortgages. On January 15, 2009, the sale of the dairy farm is finalized and the buyer paid the farmer $75,000. The fanner executed a general warranty deed. The deed contains the following covenants of title:(1) Covenant for seisin. (2) Covenant of the right to convey. (3) Covenant against encumbrances. After the buyer takes possession of the dairy farm, he learns of the son \u0080\u0094investor 1936 mortgage, which was not satisfied, and seeks monetary damages for breach of the covenant against encumbrances. Judgment should be for\",\n \"options\": {\n \"A\": \"the farmer, unless the covenantee is disturbed in his actual enjoyment of the land thereby conveyed.\",\n \"B\": \"the buyer, if the farmer knew about the mortgage to the investor but did not disclose it.\",\n \"C\": \"the buyer, because the covenant against encumbrances protects against future claims on the property.\",\n \"D\": \"the buyer, because the covenant of the right to convey was breached.\",\n \"E\": \"the farmer, because the son's mortgage to the investor was not mentioned in the deed.\",\n \"F\": \"the buyer, because the covenant against encumbrances is a guarantee to the grantee that the property is not subject to outstanding rights or interests.\",\n \"G\": \"the farmer, if the mortgage to the investor was satisfied before the sale to the buyer.\",\n \"H\": \"the buyer, because the covenant against encumbrances would be breached at the time the deed was delivered, thereby entitling the covenantee to recover damages.\",\n \"I\": \"the farmer, because the buyer did not perform a thorough title search before purchasing the property.\",\n \"J\": \"the farmer, because the covenant against encumbrances may only be breached, if at all, at the time of conveyance.\"\n }\n}
input
question.json
{
"question": "In 1888, a landowner owned a dairy farm. The landowner conveyed this real property to his son in1938. In 1953, the son conveyed the dairy farm to his friend. This deed was not recorded until after the son's death in 1957. In 1956, the son mortgaged the dairy farm to the bank. The mortgage instrument, which was recorded in 1956, recited that it was subordinate to a mortgage on the same land given by the son to an investor in 1936 and recorded in 1936. In that instrument the son purported to grant the investor a mortgage on the dairy farm. In 1979, the friend conveyed the dairy farm to a farmer. This deed was duly recorded, but did not mention any mortgage. In 2008, a buyer entered into an agreement with the farmer, whereby the farmer would convey the dairy farm in fee simple to the buyer for the sum of $75,000. The closing date was set for January 15, 2009. All of the deeds mentioned in the aforementioned transactions are general warranty deeds. In addition, this jurisdiction has a notice-type recording statute and follows a title theory for mortgages. On January 15, 2009, the sale of the dairy farm is finalized and the buyer paid the farmer $75,000. The fanner executed a general warranty deed. The deed contains the following covenants of title:(1) Covenant for seisin. (2) Covenant of the right to convey. (3) Covenant against encumbrances. After the buyer takes possession of the dairy farm, he learns of the son investor 1936 mortgage, which was not satisfied, and seeks monetary damages for breach of the covenant against encumbrances. Judgment should be for",
"options": {
"A": "the farmer, unless the covenantee is disturbed in his actual enjoyment of the land thereby conveyed.",
"B": "the buyer, if the farmer knew about the mortgage to the investor but did not disclose it.",
"C": "the buyer, because the covenant against encumbrances protects against future claims on the property.",
"D": "the buyer, because the covenant of the right to convey was breached.",
"E": "the farmer, because the son's mortgage to the investor was not mentioned in the deed.",
"F": "the buyer, because the covenant against encumbrances is a guarantee to the grantee that the property is not subject to outstanding rights or interests.",
"G": "the farmer, if the mortgage to the investor was satisfied before the sale to the buyer.",
"H": "the buyer, because the covenant against encumbrances would be breached at the time the deed was delivered, thereby entitling the covenantee to recover damages.",
"I": "the farmer, because the buyer did not perform a thorough title search before purchasing the property.",
"J": "the farmer, because the covenant against encumbrances may only be breached, if at all, at the time of conveyance."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_139/ files.
Scored by a judge this pinned commit does not carry.
▸case_140Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A federal employees union brought an action in federal court asking for an injunction preventing the U.S. Treasury Department from enforcing mandatory drug testing against its members who carried firearms or interdicted drugs in their official duties. When an employee in one of those categories sought a promotion, the government required him or her to undergo a program of providing urine specimens for analysis. The union claimed that these searches were in violation of the Fourth Amendment's right to privacy and to be free from unreasonable searches. The union lost the case in the lower federal court and in the Court of Appeals, but the United States Supreme Court agreed to hear it. What will the Supreme Court decide?\",\n \"options\": {\n \"A\": \"It is reasonable to conduct suspicionless testing of employees who are involved in drug interdiction or the handling of firearms.\",\n \"B\": \"It is reasonable to conduct suspicionless testing of employees who handle firearms, but not those involved in drug interdiction.\",\n \"C\": \"It is reasonable to conduct suspicionless testing of employees, but not as a condition for promotion.\",\n \"D\": \"The drug testing policy is constitutional if it is applied to all employees, regardless of their duties.\",\n \"E\": \"It is reasonable for a government employer to conduct suspicionless searches under any circumstances.\",\n \"F\": \"The drug testing policy is unconstitutional because it is not applied to all federal employees.\",\n \"G\": \"It is unreasonable to conduct suspicionless testing of employees without prior court approval.\",\n \"H\": \"The drug testing policy is unconstitutional because the searches are made without probable cause or suspicion.\",\n \"I\": \"The drug testing policy is constitutional because it is a condition of employment.\",\n \"J\": \"The drug testing policy is unconstitutional in that it violates the employees' legitimate expectations of privacy.\"\n }\n}
input
question.json
{
"question": "A federal employees union brought an action in federal court asking for an injunction preventing the U.S. Treasury Department from enforcing mandatory drug testing against its members who carried firearms or interdicted drugs in their official duties. When an employee in one of those categories sought a promotion, the government required him or her to undergo a program of providing urine specimens for analysis. The union claimed that these searches were in violation of the Fourth Amendment's right to privacy and to be free from unreasonable searches. The union lost the case in the lower federal court and in the Court of Appeals, but the United States Supreme Court agreed to hear it. What will the Supreme Court decide?",
"options": {
"A": "It is reasonable to conduct suspicionless testing of employees who are involved in drug interdiction or the handling of firearms.",
"B": "It is reasonable to conduct suspicionless testing of employees who handle firearms, but not those involved in drug interdiction.",
"C": "It is reasonable to conduct suspicionless testing of employees, but not as a condition for promotion.",
"D": "The drug testing policy is constitutional if it is applied to all employees, regardless of their duties.",
"E": "It is reasonable for a government employer to conduct suspicionless searches under any circumstances.",
"F": "The drug testing policy is unconstitutional because it is not applied to all federal employees.",
"G": "It is unreasonable to conduct suspicionless testing of employees without prior court approval.",
"H": "The drug testing policy is unconstitutional because the searches are made without probable cause or suspicion.",
"I": "The drug testing policy is constitutional because it is a condition of employment.",
"J": "The drug testing policy is unconstitutional in that it violates the employees' legitimate expectations of privacy."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_140/ files.
Scored by a judge this pinned commit does not carry.
▸case_141Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A young man suggested to his friend that they steal a large-screen TV from a neighbor's house. The friend wasangry with the young man and decided to use the opportunity to get even with him by having him arrested. Thefriend said he would help, and that night, he drove the young man to the neighbor's house. The young manbroke in while the friend remained outside. The friend called the police on his cell phone and then drove away. Police officers arrived at the scene just as the young man was carrying the TV out the back door. The friend is guilty of what offense in a common law jurisdiction? \",\n \"options\": {\n \"A\": \"Conspiracy.\",\n \"B\": \"Conspiracy and larceny.\",\n \"C\": \"Burglary.\",\n \"D\": \"Solicitation to commit a crime.\",\n \"E\": \"No crime.\",\n \"F\": \"Larceny.\",\n \"G\": \"Conspiracy and accessory to burglary.\",\n \"H\": \"Accessory to burglary.\",\n \"I\": \"Criminal facilitation.\",\n \"J\": \"Aiding and abetting.\"\n }\n}
input
question.json
{
"question": "A young man suggested to his friend that they steal a large-screen TV from a neighbor's house. The friend wasangry with the young man and decided to use the opportunity to get even with him by having him arrested. Thefriend said he would help, and that night, he drove the young man to the neighbor's house. The young manbroke in while the friend remained outside. The friend called the police on his cell phone and then drove away. Police officers arrived at the scene just as the young man was carrying the TV out the back door. The friend is guilty of what offense in a common law jurisdiction? ",
"options": {
"A": "Conspiracy.",
"B": "Conspiracy and larceny.",
"C": "Burglary.",
"D": "Solicitation to commit a crime.",
"E": "No crime.",
"F": "Larceny.",
"G": "Conspiracy and accessory to burglary.",
"H": "Accessory to burglary.",
"I": "Criminal facilitation.",
"J": "Aiding and abetting."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_141/ files.
Scored by a judge this pinned commit does not carry.
▸case_142Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Given the probability distribution of the random variable X in the table below, compute E(X) andVar(X). x_i Pr(X = x_i) 0 (8/27) 1 (12/27) 2 (6/27) 3 (1/27)\",\n \"options\": {\n \"A\": \"E(X) = 3/2, Var(X) = 1/4\",\n \"B\": \"E(X) = 2, Var(X) = 1/3\",\n \"C\": \"E(X) = 5/3, Var(X) = 5/9\",\n \"D\": \"E(X) = 1, Var(X) = 2/3\",\n \"E\": \"E(X) = 3/4, Var(X) = 3/8\",\n \"F\": \"E(X) = 2/3, Var(X) = 1\",\n \"G\": \"E(X) = 0.5, Var(X) = 0.5\",\n \"H\": \"E(X) = 1.5, Var(X) = 1.5\",\n \"I\": \"E(X) = 1/2, Var(X) = 1/2\",\n \"J\": \"E(X) = 1/3, Var(X) = 2\"\n }\n}
input
question.json
{
"question": "Given the probability distribution of the random variable X in the table below, compute E(X) andVar(X). x_i Pr(X = x_i) 0 (8/27) 1 (12/27) 2 (6/27) 3 (1/27)",
"options": {
"A": "E(X) = 3/2, Var(X) = 1/4",
"B": "E(X) = 2, Var(X) = 1/3",
"C": "E(X) = 5/3, Var(X) = 5/9",
"D": "E(X) = 1, Var(X) = 2/3",
"E": "E(X) = 3/4, Var(X) = 3/8",
"F": "E(X) = 2/3, Var(X) = 1",
"G": "E(X) = 0.5, Var(X) = 0.5",
"H": "E(X) = 1.5, Var(X) = 1.5",
"I": "E(X) = 1/2, Var(X) = 1/2",
"J": "E(X) = 1/3, Var(X) = 2"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_142/ files.
Scored by a judge this pinned commit does not carry.
▸case_143Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A defendant and his college roommate went home together for the holiday vacation. The defendant was to go back to college by himself. The day that the defendant was to drive back to college, his roommate asked him if he would deliver a package to one of their classmates. The defendant agreed, and the roommate gave him a small package, which the defendant placed in his glove compartment. While driving back to school, the defendant was stopped for speeding. When the state trooper asked the defendant for his license and registration, he reached into the glove compartment. As he did, the package that his roommate had given him fell onto the floor. When it hit the floor, the box broke open and a plastic envelope containing a white substance slid out. Seeing that the substance appeared to be cocaine, the state trooper arrested the defendant for possession of illegal narcotics under a state statute making it a felony to willfully possess a controlled substance. The package did, in fact, contain cocaine. If the defendant is later brought to trial for this charge, his best defense is that\",\n \"options\": {\n \"A\": \"the package didn't belong to him.\",\n \"B\": \"the cocaine was illegally seized.\",\n \"C\": \"the trooper didn't have a search warrant.\",\n \"D\": \"the defendant didn't know what was in the package.\",\n \"E\": \"the roommate didn't tell him he was carrying illegal narcotics.\",\n \"F\": \"the roommate should be the one charged, not him.\",\n \"G\": \"the substance was not tested to confirm it was cocaine.\",\n \"H\": \"the defendant didn't willingly possess the substance.\",\n \"I\": \"the cocaine was not in a recognizable form.\",\n \"J\": \"the defendant was not speeding.\"\n }\n}
input
question.json
{
"question": "A defendant and his college roommate went home together for the holiday vacation. The defendant was to go back to college by himself. The day that the defendant was to drive back to college, his roommate asked him if he would deliver a package to one of their classmates. The defendant agreed, and the roommate gave him a small package, which the defendant placed in his glove compartment. While driving back to school, the defendant was stopped for speeding. When the state trooper asked the defendant for his license and registration, he reached into the glove compartment. As he did, the package that his roommate had given him fell onto the floor. When it hit the floor, the box broke open and a plastic envelope containing a white substance slid out. Seeing that the substance appeared to be cocaine, the state trooper arrested the defendant for possession of illegal narcotics under a state statute making it a felony to willfully possess a controlled substance. The package did, in fact, contain cocaine. If the defendant is later brought to trial for this charge, his best defense is that",
"options": {
"A": "the package didn't belong to him.",
"B": "the cocaine was illegally seized.",
"C": "the trooper didn't have a search warrant.",
"D": "the defendant didn't know what was in the package.",
"E": "the roommate didn't tell him he was carrying illegal narcotics.",
"F": "the roommate should be the one charged, not him.",
"G": "the substance was not tested to confirm it was cocaine.",
"H": "the defendant didn't willingly possess the substance.",
"I": "the cocaine was not in a recognizable form.",
"J": "the defendant was not speeding."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_143/ files.
Scored by a judge this pinned commit does not carry.
▸case_144Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A man works at a tire warehouse, where his job duties consist of loading tires onto trucks. The job is exhausting, and the warehouse is always short of personnel. One day, while loading a heavy tractor tire onto a truck by himself, he experienced a sharp and excruciating pain in his lower back. He did not report the incident to his employer, but continued to work. A week later, after work, he went to the doctor for treatment. The man then sues his employer, alleging that the employer failed to provide him with adequate assistance, safety equipment, and training to load heavy tractor tires, thereby contributing to his injury. The employer alleges that the man is merely trying to obtain compensation for an old back injury, unrelated to his employment. At trial, the man's doctor testified that the man told her that his back began hurting while he was loading the tire on the truck. The doctor's testimony is\",\n \"options\": {\n \"A\": \"admissible as a statement for the purposes of medical diagnosis.\",\n \"B\": \"admissible as a statement made in the course of treatment.\",\n \"C\": \"inadmissible as a violation of the physicianpatient privilege.\",\n \"D\": \"admissible as it is a statement of fact.\",\n \"E\": \"admissible as expert testimony.\",\n \"F\": \"admissible as it is a statement made in anticipation of litigation.\",\n \"G\": \"inadmissible as it is a speculative statement.\",\n \"H\": \"inadmissible as it is a biased statement.\",\n \"I\": \"inadmissible as hearsay not within any recognized exception.\",\n \"J\": \"inadmissible as it is a self-serving statement.\"\n }\n}
input
question.json
{
"question": "A man works at a tire warehouse, where his job duties consist of loading tires onto trucks. The job is exhausting, and the warehouse is always short of personnel. One day, while loading a heavy tractor tire onto a truck by himself, he experienced a sharp and excruciating pain in his lower back. He did not report the incident to his employer, but continued to work. A week later, after work, he went to the doctor for treatment. The man then sues his employer, alleging that the employer failed to provide him with adequate assistance, safety equipment, and training to load heavy tractor tires, thereby contributing to his injury. The employer alleges that the man is merely trying to obtain compensation for an old back injury, unrelated to his employment. At trial, the man's doctor testified that the man told her that his back began hurting while he was loading the tire on the truck. The doctor's testimony is",
"options": {
"A": "admissible as a statement for the purposes of medical diagnosis.",
"B": "admissible as a statement made in the course of treatment.",
"C": "inadmissible as a violation of the physicianpatient privilege.",
"D": "admissible as it is a statement of fact.",
"E": "admissible as expert testimony.",
"F": "admissible as it is a statement made in anticipation of litigation.",
"G": "inadmissible as it is a speculative statement.",
"H": "inadmissible as it is a biased statement.",
"I": "inadmissible as hearsay not within any recognized exception.",
"J": "inadmissible as it is a self-serving statement."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_144/ files.
Scored by a judge this pinned commit does not carry.
▸case_145Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"In which of the following situations would the defendant's intoxication NOT be able to negate his criminal culpability?\",\n \"options\": {\n \"A\": \"A defendant had been drinking liquor all evening at a bar with three of his buddies. An undercover detective overheard the defendant and his buddies plot to rob the bar after closing hours. When the defendant attempted to draw a gun from his coat, he was quickly disarmed and placed under arrest by the detective. The defendant is charged with the crime of conspiracy to commit robbery.\",\n \"B\": \"After consuming a large amount of alcohol at a party, a defendant decided to set a neighbor's car on fire out of spite. He was seen by witnesses and later arrested and charged with arson.\",\n \"C\": \"A defendant had been drinking at a local pub and got into a heated argument with another patron. The argument escalated, and the defendant pulled out a pocket knife and stabbed the other patron. The defendant is arrested and charged with attempted murder.\",\n \"D\": \"A defendant had consumed a fifth of bourbon. Later that same day, he approached a victim and, brandishing a knife, told her to accompany him or he would stab her. He led the victim to his car and then ordered her to disrobe. As the victim was removing her pantyhose, she kicked the defendant in the head, temporarily dazing him. The victim then safely ran from the car. The defendant is arrested and charged with the crime of assault with the intent to commit rape.\",\n \"E\": \"A defendant spent the evening drinking at a bar. On his way home, he broke into a house, intending to steal valuables. However, he fell asleep on the couch before he could steal anything. He was discovered by the homeowners and arrested and charged with attempted robbery.\",\n \"F\": \"At his law school graduation party, a defendant drank two six-packs of beer. Around midnight, the defendant was approached by a girl, who asked him to drive her home. Although the girl was only 15 years old, she had the appearance of a woman in her mid-to-late twenties. The defendant, who had had his eye on the girl all night, quickly agreed, and he showed her the way to his car. Once inside, they engaged in sexual intercourse. The age of consent in this jurisdiction is 17 years old for females. The defendant is subsequently arrested and charged with statutory rape.\",\n \"G\": \"A defendant was drinking at a bar when he noticed a woman leave her purse unattended. He stole the purse and was later arrested and charged with theft.\",\n \"H\": \"A defendant attended a wedding reception at a hotel, where he drank several vodka daiquiris. Following the reception, the defendant engaged in a violent argument with the hotel's parking lot attendant. The defendant took a tire iron from his car and threw it at the attendant. The tire iron missed the attendant and hit a victim as he was entering the hotel. The defendant is arrested and charged with assault with the intent to commit battery.\",\n \"I\": \"A defendant had been drinking heavily at a friend's house. Later that evening, he decided to drive home despite being significantly over the legal blood alcohol limit. On the way, he ran a red light and hit a pedestrian who was crossing the street. The defendant is arrested and charged with vehicular manslaughter.\",\n \"J\": \"After a night of heavy drinking, a defendant broke into a jewelry store and stole several expensive items. He was caught on the store's surveillance cameras and later arrested and charged with burglary.\"\n }\n}
input
question.json
{
"question": "In which of the following situations would the defendant's intoxication NOT be able to negate his criminal culpability?",
"options": {
"A": "A defendant had been drinking liquor all evening at a bar with three of his buddies. An undercover detective overheard the defendant and his buddies plot to rob the bar after closing hours. When the defendant attempted to draw a gun from his coat, he was quickly disarmed and placed under arrest by the detective. The defendant is charged with the crime of conspiracy to commit robbery.",
"B": "After consuming a large amount of alcohol at a party, a defendant decided to set a neighbor's car on fire out of spite. He was seen by witnesses and later arrested and charged with arson.",
"C": "A defendant had been drinking at a local pub and got into a heated argument with another patron. The argument escalated, and the defendant pulled out a pocket knife and stabbed the other patron. The defendant is arrested and charged with attempted murder.",
"D": "A defendant had consumed a fifth of bourbon. Later that same day, he approached a victim and, brandishing a knife, told her to accompany him or he would stab her. He led the victim to his car and then ordered her to disrobe. As the victim was removing her pantyhose, she kicked the defendant in the head, temporarily dazing him. The victim then safely ran from the car. The defendant is arrested and charged with the crime of assault with the intent to commit rape.",
"E": "A defendant spent the evening drinking at a bar. On his way home, he broke into a house, intending to steal valuables. However, he fell asleep on the couch before he could steal anything. He was discovered by the homeowners and arrested and charged with attempted robbery.",
"F": "At his law school graduation party, a defendant drank two six-packs of beer. Around midnight, the defendant was approached by a girl, who asked him to drive her home. Although the girl was only 15 years old, she had the appearance of a woman in her mid-to-late twenties. The defendant, who had had his eye on the girl all night, quickly agreed, and he showed her the way to his car. Once inside, they engaged in sexual intercourse. The age of consent in this jurisdiction is 17 years old for females. The defendant is subsequently arrested and charged with statutory rape.",
"G": "A defendant was drinking at a bar when he noticed a woman leave her purse unattended. He stole the purse and was later arrested and charged with theft.",
"H": "A defendant attended a wedding reception at a hotel, where he drank several vodka daiquiris. Following the reception, the defendant engaged in a violent argument with the hotel's parking lot attendant. The defendant took a tire iron from his car and threw it at the attendant. The tire iron missed the attendant and hit a victim as he was entering the hotel. The defendant is arrested and charged with assault with the intent to commit battery.",
"I": "A defendant had been drinking heavily at a friend's house. Later that evening, he decided to drive home despite being significantly over the legal blood alcohol limit. On the way, he ran a red light and hit a pedestrian who was crossing the street. The defendant is arrested and charged with vehicular manslaughter.",
"J": "After a night of heavy drinking, a defendant broke into a jewelry store and stole several expensive items. He was caught on the store's surveillance cameras and later arrested and charged with burglary."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_145/ files.
Scored by a judge this pinned commit does not carry.
▸case_146Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Find the general solution of Bessel's equation of order one.\",\n \"options\": {\n \"A\": \"y(x) = c_1 * sin(x) + c_2 * cos(x)\",\n \"B\": \"y(x) = c_1 * y_1(x)^2 + c_2 * y_2(x)^2\",\n \"C\": \"y(x) = c_1y_1(x) + c_2y_2(x)\",\n \"D\": \"y(x) = c_1 * x * y_1(x) + c_2 / (x * y_2(x))\",\n \"E\": \"y(x) = c_1 * y_1(x) / (c_2 * y_2(x))\",\n \"F\": \"y(x) = c_1 * ln|x| + c_2 * x^2\",\n \"G\": \"y(x) = c_1y_1(x) * c_2y_2(x)\",\n \"H\": \"y(x) = c_1/y_1(x) + c_2/y_2(x)\",\n \"I\": \"y(x) = c_1y_1(x) - c_2y_2(x)\",\n \"J\": \"y(x) = c_1 * e^(x) + c_2 * e^(-x)\"\n }\n}
input
question.json
{
"question": "Find the general solution of Bessel's equation of order one.",
"options": {
"A": "y(x) = c_1 * sin(x) + c_2 * cos(x)",
"B": "y(x) = c_1 * y_1(x)^2 + c_2 * y_2(x)^2",
"C": "y(x) = c_1y_1(x) + c_2y_2(x)",
"D": "y(x) = c_1 * x * y_1(x) + c_2 / (x * y_2(x))",
"E": "y(x) = c_1 * y_1(x) / (c_2 * y_2(x))",
"F": "y(x) = c_1 * ln|x| + c_2 * x^2",
"G": "y(x) = c_1y_1(x) * c_2y_2(x)",
"H": "y(x) = c_1/y_1(x) + c_2/y_2(x)",
"I": "y(x) = c_1y_1(x) - c_2y_2(x)",
"J": "y(x) = c_1 * e^(x) + c_2 * e^(-x)"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_146/ files.
Scored by a judge this pinned commit does not carry.
▸case_147Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"If 5 watts/meter^2 is thePoyntingvector of a plane wave traveling in free space, find its average energy density.\",\n \"options\": {\n \"A\": \"(1/2) erg/meter^3\",\n \"B\": \"(1/3)erg/meter^3\",\n \"C\": \"(1/8) erg/meter^3\",\n \"D\": \"(3/6) erg/meter^3\",\n \"E\": \"(1/5) erg/meter^3\",\n \"F\": \"(1/4)erg/meter^3\",\n \"G\": \"(2/4) erg/meter^3\",\n \"H\": \"(3/4) erg/meter^3\",\n \"I\": \"(1/6)erg/meter^3\",\n \"J\": \"(2/6)erg/meter^3\"\n }\n}
input
question.json
{
"question": "If 5 watts/meter^2 is thePoyntingvector of a plane wave traveling in free space, find its average energy density.",
"options": {
"A": "(1/2) erg/meter^3",
"B": "(1/3)erg/meter^3",
"C": "(1/8) erg/meter^3",
"D": "(3/6) erg/meter^3",
"E": "(1/5) erg/meter^3",
"F": "(1/4)erg/meter^3",
"G": "(2/4) erg/meter^3",
"H": "(3/4) erg/meter^3",
"I": "(1/6)erg/meter^3",
"J": "(2/6)erg/meter^3"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_147/ files.
Scored by a judge this pinned commit does not carry.
▸case_148Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Determine the mass and specific volume of argon gas in a vessel at 150kPaand 20°C. The vessel is spherical and has a radius of 5m.\",\n \"options\": {\n \"A\": \"mass = 1350 kg, specific volume = 0.47 m^3/kg\",\n \"B\": \"mass = 1288 kg, specific volume = 0.4065 m^3/kg\",\n \"C\": \"mass = 1200 kg, specific volume = 0.4 m^3/kg\",\n \"D\": \"mass = 1450 kg, specific volume = 0.38 m^3/kg\",\n \"E\": \"mass = 1300 kg, specific volume = 0.5 m^3/kg\",\n \"F\": \"mass = 1250 kg, specific volume = 0.48 m^3/kg\",\n \"G\": \"mass = 1500 kg, specific volume = 0.35 m^3/kg\",\n \"H\": \"mass = 1400 kg, specific volume = 0.45 m^3/kg\",\n \"I\": \"mass = 1150 kg, specific volume = 0.42 m^3/kg\",\n \"J\": \"mass = 1100 kg, specific volume = 0.6 m^3/kg\"\n }\n}
input
question.json
{
"question": "Determine the mass and specific volume of argon gas in a vessel at 150kPaand 20°C. The vessel is spherical and has a radius of 5m.",
"options": {
"A": "mass = 1350 kg, specific volume = 0.47 m^3/kg",
"B": "mass = 1288 kg, specific volume = 0.4065 m^3/kg",
"C": "mass = 1200 kg, specific volume = 0.4 m^3/kg",
"D": "mass = 1450 kg, specific volume = 0.38 m^3/kg",
"E": "mass = 1300 kg, specific volume = 0.5 m^3/kg",
"F": "mass = 1250 kg, specific volume = 0.48 m^3/kg",
"G": "mass = 1500 kg, specific volume = 0.35 m^3/kg",
"H": "mass = 1400 kg, specific volume = 0.45 m^3/kg",
"I": "mass = 1150 kg, specific volume = 0.42 m^3/kg",
"J": "mass = 1100 kg, specific volume = 0.6 m^3/kg"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_148/ files.
Scored by a judge this pinned commit does not carry.
▸case_149Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"The energy stored in the magnetic field in a solenoid 30 cm long and 3 cm diameter wound with 1000 turns of wire carrying a current at 10 amp, is\",\n \"options\": {\n \"A\": \"0.015 joule.\",\n \"B\": \"15 joule.\",\n \"C\": \"0.0015 joule.\",\n \"D\": \"1.5 joule.\",\n \"E\": \"0.15 millijoule.\",\n \"F\": \"1.15 joule.\",\n \"G\": \"0.15 joule.\",\n \"H\": \"15 millijoule.\"\n }\n}
input
question.json
{
"question": "The energy stored in the magnetic field in a solenoid 30 cm long and 3 cm diameter wound with 1000 turns of wire carrying a current at 10 amp, is",
"options": {
"A": "0.015 joule.",
"B": "15 joule.",
"C": "0.0015 joule.",
"D": "1.5 joule.",
"E": "0.15 millijoule.",
"F": "1.15 joule.",
"G": "0.15 joule.",
"H": "15 millijoule."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_149/ files.
Scored by a judge this pinned commit does not carry.
▸case_150Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Arrests may occasionally be made without a warrant. Which of the following is not one of those circumstances?\",\n \"options\": {\n \"A\": \"An officer may arrest for a misdemeanor that is not a breach of peace and committed out of arresting party's presence.\",\n \"B\": \"An officer must have probable cause that a felony has occurred and that the arrested person committed it.\",\n \"C\": \"An officer may arrest for a misdemeanor committed in their presence without any evidence.\",\n \"D\": \"An officer may arrest a person if they are suspected of committing a misdemeanor that has not yet occurred.\",\n \"E\": \"An officer may arrest a person if they have committed a felony, regardless of the officer's presence during the crime.\",\n \"F\": \"An officer may arrest if he suspects a person will commit a felony in the future.\",\n \"G\": \"A misdemeanor has occurred, and an officer must believe that the person he has arrested committed it.\",\n \"H\": \"An officer may arrest for a misdemeanor that is a breach of peace and committed in arresting party's presence.\",\n \"I\": \"An officer may arrest a person based on a tip from an unverified source.\",\n \"J\": \"An officer may arrest for a felony committed in his presence.\"\n }\n}
input
question.json
{
"question": "Arrests may occasionally be made without a warrant. Which of the following is not one of those circumstances?",
"options": {
"A": "An officer may arrest for a misdemeanor that is not a breach of peace and committed out of arresting party's presence.",
"B": "An officer must have probable cause that a felony has occurred and that the arrested person committed it.",
"C": "An officer may arrest for a misdemeanor committed in their presence without any evidence.",
"D": "An officer may arrest a person if they are suspected of committing a misdemeanor that has not yet occurred.",
"E": "An officer may arrest a person if they have committed a felony, regardless of the officer's presence during the crime.",
"F": "An officer may arrest if he suspects a person will commit a felony in the future.",
"G": "A misdemeanor has occurred, and an officer must believe that the person he has arrested committed it.",
"H": "An officer may arrest for a misdemeanor that is a breach of peace and committed in arresting party's presence.",
"I": "An officer may arrest a person based on a tip from an unverified source.",
"J": "An officer may arrest for a felony committed in his presence."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_150/ files.
Scored by a judge this pinned commit does not carry.
▸case_151Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A pedestrian was crossing the street when he was hit by a car. The pedestrian suffered a neck injury. The driver of the car that hit the pedestrian had auto insurance. A claims adjuster from that insurance company contacted the pedestrian and discussed the accident with him. The insurance adjuster said, \\\"You're not actually entitled to any compensation for these injuries, under the laws of this state; but I feel bad for you, and I'm a nice guy, so I'm willing to approve a $1,000 payment to you. \\\" The insurance adjuster knew that, in fact, the pedestrian would be entitled to compensation under the state's tort law. The pedestrian rejected the offer, retained an attorney, and wound up winning a $550,000 jury verdict on negligence claims against the driver. If the pedestrian asserts a claim against the insurance adjuster for misrepresentation, will the pedestrian prevail?\",\n \"options\": {\n \"A\": \"No, because the insurance adjuster's statements did not cause the pedestrian any monetary loss.\",\n \"B\": \"No, because the insurance adjuster was not acting in a legal capacity when discussing the case with the pedestrian.\",\n \"C\": \"Yes, because the adjuster intentionally misled the pedestrian about his legal rights.\",\n \"D\": \"No, because the pedestrian did not rely on the adjuster's statement in deciding to reject the offer.\",\n \"E\": \"No, because the pedestrian won the case and received more than the initial offer.\",\n \"F\": \"Yes, because the insurance adjuster's $1,000 settlement offer was grossly inadequate.\",\n \"G\": \"Yes, because the insurance adjuster knew that laws of the state provided for recovery in such a case.\",\n \"H\": \"Yes, because the pedestrian was emotionally distressed due to the adjuster's misrepresentation.\",\n \"I\": \"No, because the insurance adjuster did not have a fiduciary relationship requiring him to protect the pedestrian's interests.\",\n \"J\": \"Yes, because the insurance adjuster's statement was a direct violation of the pedestrian's rights.\"\n }\n}
input
question.json
{
"question": "A pedestrian was crossing the street when he was hit by a car. The pedestrian suffered a neck injury. The driver of the car that hit the pedestrian had auto insurance. A claims adjuster from that insurance company contacted the pedestrian and discussed the accident with him. The insurance adjuster said, \"You're not actually entitled to any compensation for these injuries, under the laws of this state; but I feel bad for you, and I'm a nice guy, so I'm willing to approve a $1,000 payment to you. \" The insurance adjuster knew that, in fact, the pedestrian would be entitled to compensation under the state's tort law. The pedestrian rejected the offer, retained an attorney, and wound up winning a $550,000 jury verdict on negligence claims against the driver. If the pedestrian asserts a claim against the insurance adjuster for misrepresentation, will the pedestrian prevail?",
"options": {
"A": "No, because the insurance adjuster's statements did not cause the pedestrian any monetary loss.",
"B": "No, because the insurance adjuster was not acting in a legal capacity when discussing the case with the pedestrian.",
"C": "Yes, because the adjuster intentionally misled the pedestrian about his legal rights.",
"D": "No, because the pedestrian did not rely on the adjuster's statement in deciding to reject the offer.",
"E": "No, because the pedestrian won the case and received more than the initial offer.",
"F": "Yes, because the insurance adjuster's $1,000 settlement offer was grossly inadequate.",
"G": "Yes, because the insurance adjuster knew that laws of the state provided for recovery in such a case.",
"H": "Yes, because the pedestrian was emotionally distressed due to the adjuster's misrepresentation.",
"I": "No, because the insurance adjuster did not have a fiduciary relationship requiring him to protect the pedestrian's interests.",
"J": "Yes, because the insurance adjuster's statement was a direct violation of the pedestrian's rights."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_151/ files.
Scored by a judge this pinned commit does not carry.
▸case_152Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A defendant went to a party at his friend's house. While he was there, the defendant drank a lot of beers and countless shots. The defendant eventually stumbled into the bathroom where he spotted a purse lying on the floor under the sink. Wondering vaguely if it might contain more beer, he opened it up and found his friend's wallet. He grabbed some money from the wallet and left the bathroom. As he was descending the stairs with the money in his hand, his friend spotted him. She was initially concerned he might tumble down the stairs but upon seeing the money in his hand, she asked him where he got it. The defendant responded, \\\"Ha! Your money or your life!\\\" Not taking him seriously, the friend let him pass by. It wasn't until later that she realized that the defendant had taken the money from her wallet. The defendant should be charged with which of the following crimes?\",\n \"options\": {\n \"A\": \"Larceny.\",\n \"B\": \"Larceny, burglary, and attempted extortion.\",\n \"C\": \"Robbery, extortion, and assault.\",\n \"D\": \"Robbery.\",\n \"E\": \"Robbery and extortion.\",\n \"F\": \"Larceny and attempted extortion.\",\n \"G\": \"Extortion.\",\n \"H\": \"Assault.\",\n \"I\": \"Burglary.\",\n \"J\": \"Larceny and burglary.\"\n }\n}
input
question.json
{
"question": "A defendant went to a party at his friend's house. While he was there, the defendant drank a lot of beers and countless shots. The defendant eventually stumbled into the bathroom where he spotted a purse lying on the floor under the sink. Wondering vaguely if it might contain more beer, he opened it up and found his friend's wallet. He grabbed some money from the wallet and left the bathroom. As he was descending the stairs with the money in his hand, his friend spotted him. She was initially concerned he might tumble down the stairs but upon seeing the money in his hand, she asked him where he got it. The defendant responded, \"Ha! Your money or your life!\" Not taking him seriously, the friend let him pass by. It wasn't until later that she realized that the defendant had taken the money from her wallet. The defendant should be charged with which of the following crimes?",
"options": {
"A": "Larceny.",
"B": "Larceny, burglary, and attempted extortion.",
"C": "Robbery, extortion, and assault.",
"D": "Robbery.",
"E": "Robbery and extortion.",
"F": "Larceny and attempted extortion.",
"G": "Extortion.",
"H": "Assault.",
"I": "Burglary.",
"J": "Larceny and burglary."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_152/ files.
Scored by a judge this pinned commit does not carry.
▸case_153Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"(i) How much energy is used by a 1,500-watt heater in 8 hr? (ii) A 12-kw load is supplied from a line that has a resistance of 0.1 ohm in each conductor. Find the line loss in watts when the load is supplied at (a) 120 volts and (b) 240 volts.\",\n \"options\": {\n \"A\": \"9kwhr, 1440 watts, 720 watts\",\n \"B\": \"14kwhr, 2100 watts, 300 watts\",\n \"C\": \"11kwhr, 1800 watts, 450 watts\",\n \"D\": \"15kwhr, 2500 watts, 400 watts\",\n \"E\": \"12kwhr, 2000 watts, 500 watts\",\n \"F\": \"10kwhr, 1500 watts, 600 watts\",\n \"G\": \"11.5kwhr, 1900 watts, 520 watts\",\n \"H\": \"12kwhr, 2200 watts, 440 watts\",\n \"I\": \"12.5kwhr, 1700 watts, 560 watts\",\n \"J\": \"13kwhr, 1600 watts, 480 watts\"\n }\n}
input
question.json
{
"question": "(i) How much energy is used by a 1,500-watt heater in 8 hr? (ii) A 12-kw load is supplied from a line that has a resistance of 0.1 ohm in each conductor. Find the line loss in watts when the load is supplied at (a) 120 volts and (b) 240 volts.",
"options": {
"A": "9kwhr, 1440 watts, 720 watts",
"B": "14kwhr, 2100 watts, 300 watts",
"C": "11kwhr, 1800 watts, 450 watts",
"D": "15kwhr, 2500 watts, 400 watts",
"E": "12kwhr, 2000 watts, 500 watts",
"F": "10kwhr, 1500 watts, 600 watts",
"G": "11.5kwhr, 1900 watts, 520 watts",
"H": "12kwhr, 2200 watts, 440 watts",
"I": "12.5kwhr, 1700 watts, 560 watts",
"J": "13kwhr, 1600 watts, 480 watts"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_153/ files.
Scored by a judge this pinned commit does not carry.
▸case_154Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Which of the following propositions would be rejected by those who describe themselves as critical legal theorists?\",\n \"options\": {\n \"A\": \"Law is unstable.\",\n \"B\": \"Law is determinate.\",\n \"C\": \"Law reflects economic power.\",\n \"D\": \"Law is politics.\",\n \"E\": \"Law is a social construct.\"\n }\n}
input
question.json
{
"question": "Which of the following propositions would be rejected by those who describe themselves as critical legal theorists?",
"options": {
"A": "Law is unstable.",
"B": "Law is determinate.",
"C": "Law reflects economic power.",
"D": "Law is politics.",
"E": "Law is a social construct."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_154/ files.
Scored by a judge this pinned commit does not carry.
▸case_155Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Two years ago, a developer conveyed title to a lot to a woman by warranty deed. The woman purchased an owner's policy of title insurance before the an owner's policy of title insurance before the closing. Because of an error by the title insurance company, the title commitment and title policy failed to list a county tax lien encumbering the lot as an exception to coverage. Last year, the woman conveyed the lot for a bargain price by a quitclaim deed to a friend, who paid cash, did not conduct a title search, and did not purchase title insurance. Subsequently, the county began proceedings to foreclose the tax lien, and the friend filed a claim with the woman's title insurance company, demanding that the company reimburse her for the tax lien amount. Is the title insurance company obligated to pay the friend's claim?\",\n \"options\": {\n \"A\": \"No, because the friend did not conduct a title search before purchasing the lot.\",\n \"B\": \"No, because the friend did not purchase her own title insurance.\",\n \"C\": \"No, because the title insurance policy expired when the woman conveyed the lot to the friend.\",\n \"D\": \"Yes, because the tax lien was not listed as an exception on the title insurance policy.\",\n \"E\": \"No, because the woman no longer owns the property, and thus her title insurance is no longer valid.\",\n \"F\": \"Yes, because the woman used a quitclaim deed, which means all encumbrances should transfer to the new owner.\",\n \"G\": \"No, because the friend is not a named insured on the title insurance policy.\",\n \"H\": \"Yes, because the friend paid cash, making her eligible for claims.\",\n \"I\": \"Yes, because the woman is liable to the friend for breach of the covenant against encumbrances, and the title policy should cover the friend's potential loss.\",\n \"J\": \"Yes, because the title insurance company made an error in not listing the tax lien.\"\n }\n}
input
question.json
{
"question": "Two years ago, a developer conveyed title to a lot to a woman by warranty deed. The woman purchased an owner's policy of title insurance before the an owner's policy of title insurance before the closing. Because of an error by the title insurance company, the title commitment and title policy failed to list a county tax lien encumbering the lot as an exception to coverage. Last year, the woman conveyed the lot for a bargain price by a quitclaim deed to a friend, who paid cash, did not conduct a title search, and did not purchase title insurance. Subsequently, the county began proceedings to foreclose the tax lien, and the friend filed a claim with the woman's title insurance company, demanding that the company reimburse her for the tax lien amount. Is the title insurance company obligated to pay the friend's claim?",
"options": {
"A": "No, because the friend did not conduct a title search before purchasing the lot.",
"B": "No, because the friend did not purchase her own title insurance.",
"C": "No, because the title insurance policy expired when the woman conveyed the lot to the friend.",
"D": "Yes, because the tax lien was not listed as an exception on the title insurance policy.",
"E": "No, because the woman no longer owns the property, and thus her title insurance is no longer valid.",
"F": "Yes, because the woman used a quitclaim deed, which means all encumbrances should transfer to the new owner.",
"G": "No, because the friend is not a named insured on the title insurance policy.",
"H": "Yes, because the friend paid cash, making her eligible for claims.",
"I": "Yes, because the woman is liable to the friend for breach of the covenant against encumbrances, and the title policy should cover the friend's potential loss.",
"J": "Yes, because the title insurance company made an error in not listing the tax lien."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_155/ files.
Scored by a judge this pinned commit does not carry.
▸case_156Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"In an action for malpractice, a plaintiff sued a doctor, claiming that the latter failed to properly diagnose his emphysema condition. The plaintiff contends that he went to the doctor for a medical examination after experiencing chest pains and shortness of breath. However, the doctor found no evidence of the disease and advised him that he was probably suffering from indigestion. Thereafter, the plaintiff was examined by another doctor who discovered that the plaintiff in fact was suffering from a chronic case of emphysema. At trial, the plaintiff calls the other doctor as his first witness. The other doctor proposes to testify the x-rays of the plaintiff's lungs showed that his respiratory organs were marked by distension, which was characteristic of emphysema. Upon objection by the defendant's attorney, the testimony should be\",\n \"options\": {\n \"A\": \"admitted, because a doctor is properly qualified as an expert in medical matters.\",\n \"B\": \"admitted, because the other doctor followed accepted medical practice in arriving at his opinion.\",\n \"C\": \"excluded, because his opinion is based upon facts not in evidence.\",\n \"D\": \"excluded, because the x-rays are the best evidence to show the distension of the plaintiff's lungs.\"\n }\n}
input
question.json
{
"question": "In an action for malpractice, a plaintiff sued a doctor, claiming that the latter failed to properly diagnose his emphysema condition. The plaintiff contends that he went to the doctor for a medical examination after experiencing chest pains and shortness of breath. However, the doctor found no evidence of the disease and advised him that he was probably suffering from indigestion. Thereafter, the plaintiff was examined by another doctor who discovered that the plaintiff in fact was suffering from a chronic case of emphysema. At trial, the plaintiff calls the other doctor as his first witness. The other doctor proposes to testify the x-rays of the plaintiff's lungs showed that his respiratory organs were marked by distension, which was characteristic of emphysema. Upon objection by the defendant's attorney, the testimony should be",
"options": {
"A": "admitted, because a doctor is properly qualified as an expert in medical matters.",
"B": "admitted, because the other doctor followed accepted medical practice in arriving at his opinion.",
"C": "excluded, because his opinion is based upon facts not in evidence.",
"D": "excluded, because the x-rays are the best evidence to show the distension of the plaintiff's lungs."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_156/ files.
Scored by a judge this pinned commit does not carry.
▸case_157Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A motorist's negligent driving seriously injured another driver. The motorist was cited for traveling in the wrong lane and paid a fine. The motorist had a policy limit of only $15,000 for third party liability on his insurance policy. After furnishing the motorist's insurer with a full medical report, all medical bills, medical records, and other required proofs of damages, the injured claimant demanded the policy limits. The claimant gave notice that a bad faith claim would be filed if the policy limits were not paid immediately. The insurance company refused payment without explanation. A jury awarded the injured claimant $200,000. The injured claimant obtained a signed assignment from the negligent motorist giving to the claimant any rights that the motorist may have had against his insurer. Will the court likely compel the insurance company to pay the full amount of the jury verdict to injured person under these facts?\",\n \"options\": {\n \"A\": \"The court will not compel the insurance company to pay because the motorist was at fault, not the insurance company.\",\n \"B\": \"Bad faith is personal to the insured party and his insurer \u0080\u0093 the assignment was invalid, and the injured person has no claim to compel the insurer to pay.\",\n \"C\": \"The insurance company is liable to pay the full amount as they failed to act in good faith and fair dealing, regardless of the policy limit.\",\n \"D\": \"The insurance company breached the implied covenant of good faith and fair dealing that is implicit in every insurance contract and must pay the full amount of the judgment.\",\n \"E\": \"The injured person can only claim up to the policy limit from the insurer, and must seek the remaining amount from the motorist directly.\",\n \"F\": \"The injured party must enforce his entire judgment against the motorist and may never pursue the insurer for that amount.\",\n \"G\": \"The insurance company can only be compelled to pay if they were given a reasonable opportunity to settle the claim within the policy limits.\",\n \"H\": \"The policy limit is $15,000 and that is all that can be collected by the injured person \u0080\u0093 there is no action for bad faith on an insurance contract.\",\n \"I\": \"The insurance company can be compelled to pay the full amount only if they were given prior warning about the potential for a bad faith claim.\"\n }\n}
input
question.json
{
"question": "A motorist's negligent driving seriously injured another driver. The motorist was cited for traveling in the wrong lane and paid a fine. The motorist had a policy limit of only $15,000 for third party liability on his insurance policy. After furnishing the motorist's insurer with a full medical report, all medical bills, medical records, and other required proofs of damages, the injured claimant demanded the policy limits. The claimant gave notice that a bad faith claim would be filed if the policy limits were not paid immediately. The insurance company refused payment without explanation. A jury awarded the injured claimant $200,000. The injured claimant obtained a signed assignment from the negligent motorist giving to the claimant any rights that the motorist may have had against his insurer. Will the court likely compel the insurance company to pay the full amount of the jury verdict to injured person under these facts?",
"options": {
"A": "The court will not compel the insurance company to pay because the motorist was at fault, not the insurance company.",
"B": "Bad faith is personal to the insured party and his insurer the assignment was invalid, and the injured person has no claim to compel the insurer to pay.",
"C": "The insurance company is liable to pay the full amount as they failed to act in good faith and fair dealing, regardless of the policy limit.",
"D": "The insurance company breached the implied covenant of good faith and fair dealing that is implicit in every insurance contract and must pay the full amount of the judgment.",
"E": "The injured person can only claim up to the policy limit from the insurer, and must seek the remaining amount from the motorist directly.",
"F": "The injured party must enforce his entire judgment against the motorist and may never pursue the insurer for that amount.",
"G": "The insurance company can only be compelled to pay if they were given a reasonable opportunity to settle the claim within the policy limits.",
"H": "The policy limit is $15,000 and that is all that can be collected by the injured person there is no action for bad faith on an insurance contract.",
"I": "The insurance company can be compelled to pay the full amount only if they were given prior warning about the potential for a bad faith claim."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_157/ files.
Scored by a judge this pinned commit does not carry.
▸case_158Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"In 1981, a devoted conservationist, was the owner of a 100-acre tract of undeveloped land. In that year, the conservationist conveyed the tract \\\"to my nephew in fee simple, provided, however, that the grantee agrees that neither he nor his heirs or assigns shall ever use the property for any commercial purpose. If any portion of said tract is used for other than residential purposes, then the grantor or his successors in interest may re-enter as of the grantor's former estate. \\\" This deed was properly recorded. The nephew died intestate in 1999, survived by his wife. The conservationist died in 2002, survived by his two daughters, his only heirs. During the period between 1981 and 2007, the spreading development from a nearby city began to engulf the tract. Though still undeveloped, the tract became surrounded by office buildings, shopping malls, and other commercial edifices. In 2009, the wife executed and delivered to a developer a fee simple conveyance of the tract, which the developer immediately recorded. The deed did not contain any reference to the restriction noted above. After the developer acquired title to the tract, he commenced construction of a hotel complex on a portion of the tract that bordered an apartment building. The applicable recording statute in effect in this jurisdiction provides, in part, \\\"No deed or other instrument in writing, not recorded in accordance with this statute, shall affect the title or rights to, in any real estate, or any devisee or purchaser in good faith, without knowledge of the existence of such unrecorded instruments. \\\"If one of the daughters brings suit to enjoin the developer from constructing the hotel, the plaintiff will most likely\",\n \"options\": {\n \"A\": \"lose, because the developer was a bona fide purchaser for value without notice of the restriction.\",\n \"B\": \"win, because either daughter has the right of re-entry for condition broken.\",\n \"C\": \"win, because the restriction on commercial use was recorded and thus the developer should have been aware of it.\",\n \"D\": \"lose, because a common development scheme had been established for the entire tract.\",\n \"E\": \"win, because either daughter's right to the tract vested immediately upon the developer's construction of the hotel complex.\",\n \"F\": \"lose, because the restriction was not included in the deed from the wife to the developer.\",\n \"G\": \"lose, because the wife had the right to sell the property without any restrictions.\",\n \"H\": \"win, because the developer violated the deed's prohibition against commercial use.\",\n \"I\": \"win, because the daughters, as the conservationist's only heirs, received a valid possibility of reverter from their father.\",\n \"J\": \"lose, because the restriction on the use of the property is unenforceable.\"\n }\n}
input
question.json
{
"question": "In 1981, a devoted conservationist, was the owner of a 100-acre tract of undeveloped land. In that year, the conservationist conveyed the tract \"to my nephew in fee simple, provided, however, that the grantee agrees that neither he nor his heirs or assigns shall ever use the property for any commercial purpose. If any portion of said tract is used for other than residential purposes, then the grantor or his successors in interest may re-enter as of the grantor's former estate. \" This deed was properly recorded. The nephew died intestate in 1999, survived by his wife. The conservationist died in 2002, survived by his two daughters, his only heirs. During the period between 1981 and 2007, the spreading development from a nearby city began to engulf the tract. Though still undeveloped, the tract became surrounded by office buildings, shopping malls, and other commercial edifices. In 2009, the wife executed and delivered to a developer a fee simple conveyance of the tract, which the developer immediately recorded. The deed did not contain any reference to the restriction noted above. After the developer acquired title to the tract, he commenced construction of a hotel complex on a portion of the tract that bordered an apartment building. The applicable recording statute in effect in this jurisdiction provides, in part, \"No deed or other instrument in writing, not recorded in accordance with this statute, shall affect the title or rights to, in any real estate, or any devisee or purchaser in good faith, without knowledge of the existence of such unrecorded instruments. \"If one of the daughters brings suit to enjoin the developer from constructing the hotel, the plaintiff will most likely",
"options": {
"A": "lose, because the developer was a bona fide purchaser for value without notice of the restriction.",
"B": "win, because either daughter has the right of re-entry for condition broken.",
"C": "win, because the restriction on commercial use was recorded and thus the developer should have been aware of it.",
"D": "lose, because a common development scheme had been established for the entire tract.",
"E": "win, because either daughter's right to the tract vested immediately upon the developer's construction of the hotel complex.",
"F": "lose, because the restriction was not included in the deed from the wife to the developer.",
"G": "lose, because the wife had the right to sell the property without any restrictions.",
"H": "win, because the developer violated the deed's prohibition against commercial use.",
"I": "win, because the daughters, as the conservationist's only heirs, received a valid possibility of reverter from their father.",
"J": "lose, because the restriction on the use of the property is unenforceable."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_158/ files.
Scored by a judge this pinned commit does not carry.
▸case_159Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A defendant and a victim were roommates at college. The defendant was playing the role of a serial killer in the school play and asked the victim if she could leave their apartment for a few hours while she rehearsed. The victim agreed but returned to the apartment shortly thereafter because she was curious and wanted to see the defendant rehearse. The victim quietly came back into the apartment and hid behind some curtains in the dining room in order to watch the defendant rehearse. In her role, the defendant earned a large butcher's knife, with which she stabbed the curtains at her \\\"victim. \\\" Unknown to the defendant, the victim, who was hiding behind the curtain, was stabbed in the chest and killed. If the defendant is prosecuted for the victim's death, she should be found\",\n \"options\": {\n \"A\": \"guilty of attempted murder.\",\n \"B\": \"guilty of manslaughter.\",\n \"C\": \"not guilty.\",\n \"D\": \"guilty of battery.\",\n \"E\": \"guilty of negligent homicide.\",\n \"F\": \"guilty of murder.\",\n \"G\": \"guilty of reckless endangerment.\",\n \"H\": \"guilty of second-degree murder.\",\n \"I\": \"guilty of first-degree murder.\",\n \"J\": \"guilty of aggravated assault.\"\n }\n}
input
question.json
{
"question": "A defendant and a victim were roommates at college. The defendant was playing the role of a serial killer in the school play and asked the victim if she could leave their apartment for a few hours while she rehearsed. The victim agreed but returned to the apartment shortly thereafter because she was curious and wanted to see the defendant rehearse. The victim quietly came back into the apartment and hid behind some curtains in the dining room in order to watch the defendant rehearse. In her role, the defendant earned a large butcher's knife, with which she stabbed the curtains at her \"victim. \" Unknown to the defendant, the victim, who was hiding behind the curtain, was stabbed in the chest and killed. If the defendant is prosecuted for the victim's death, she should be found",
"options": {
"A": "guilty of attempted murder.",
"B": "guilty of manslaughter.",
"C": "not guilty.",
"D": "guilty of battery.",
"E": "guilty of negligent homicide.",
"F": "guilty of murder.",
"G": "guilty of reckless endangerment.",
"H": "guilty of second-degree murder.",
"I": "guilty of first-degree murder.",
"J": "guilty of aggravated assault."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_159/ files.
Scored by a judge this pinned commit does not carry.
▸case_160Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Do third States enjoy any fishing rights within the Exclusive Economic Zone of another State?\",\n \"options\": {\n \"A\": \"Third States may never have access to fisheries within another State's EEZ\",\n \"B\": \"Third States are free to fish within another EEZ, except from certain designated areas\",\n \"C\": \"When the coastal State cannot harvest the 'total allowable catch', the coastal State is to give other State access to that surplus with priority to be given to developing and land-locked States\",\n \"D\": \"The coastal State must share all of its fish resources with all other States\",\n \"E\": \"Third States have access to fisheries within its EEZ only during a specific season\",\n \"F\": \"The coastal State is to give access to fisheries within its EEZ only to States with which it has diplomatic relations\",\n \"G\": \"Third States have unlimited fishing rights within the EEZ of any State\",\n \"H\": \"Third States can only fish within another State's EEZ with a special permit\",\n \"I\": \"The coastal State is to give access to fisheries within its EEZ only to neighbouring States\",\n \"J\": \"Third States can only fish within another State's EEZ if they have a mutual agreement\"\n }\n}
input
question.json
{
"question": "Do third States enjoy any fishing rights within the Exclusive Economic Zone of another State?",
"options": {
"A": "Third States may never have access to fisheries within another State's EEZ",
"B": "Third States are free to fish within another EEZ, except from certain designated areas",
"C": "When the coastal State cannot harvest the 'total allowable catch', the coastal State is to give other State access to that surplus with priority to be given to developing and land-locked States",
"D": "The coastal State must share all of its fish resources with all other States",
"E": "Third States have access to fisheries within its EEZ only during a specific season",
"F": "The coastal State is to give access to fisheries within its EEZ only to States with which it has diplomatic relations",
"G": "Third States have unlimited fishing rights within the EEZ of any State",
"H": "Third States can only fish within another State's EEZ with a special permit",
"I": "The coastal State is to give access to fisheries within its EEZ only to neighbouring States",
"J": "Third States can only fish within another State's EEZ if they have a mutual agreement"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_160/ files.
Scored by a judge this pinned commit does not carry.
▸case_161Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Carbon dioxide enters a heater at 995lbf/in^2, 80°F and is heated until its temperature at the exit is 900°F. Assuming the carbon dioxide to have undergone a constant pressure, steady-flow process calculate the heating required per pound of CO_2.\",\n \"options\": {\n \"A\": \"265.89 Btu/lbm\",\n \"B\": \"230.58 Btu/lbm\",\n \"C\": \"290.76 Btu/lbm\",\n \"D\": \"320.45 Btu/lbm\",\n \"E\": \"190.22 Btu/lbm\",\n \"F\": \"250.12 Btu/lbm\",\n \"G\": \"150.75 Btu/lbm\",\n \"H\": \"180.50 Btu/lbm\",\n \"I\": \"209.34 Btu/lbm\",\n \"J\": \"275.00 Btu/lbm\"\n }\n}
input
question.json
{
"question": "Carbon dioxide enters a heater at 995lbf/in^2, 80°F and is heated until its temperature at the exit is 900°F. Assuming the carbon dioxide to have undergone a constant pressure, steady-flow process calculate the heating required per pound of CO_2.",
"options": {
"A": "265.89 Btu/lbm",
"B": "230.58 Btu/lbm",
"C": "290.76 Btu/lbm",
"D": "320.45 Btu/lbm",
"E": "190.22 Btu/lbm",
"F": "250.12 Btu/lbm",
"G": "150.75 Btu/lbm",
"H": "180.50 Btu/lbm",
"I": "209.34 Btu/lbm",
"J": "275.00 Btu/lbm"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_161/ files.
Scored by a judge this pinned commit does not carry.
▸case_162Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A pseudoplastic-non-newtonian fluid is flowing through a schedule 40 standard steel pipe of 1(1/4) in. dia. at a rate of 15 gal/hr. Determine the pressure drop per foot of pipe. Properties of the fluid: density (\\\\rho) = 60lbm/ft^3; viscosity (\\\\mu_ap) = 22500 poise.\",\n \"options\": {\n \"A\": \"42.3 psi/ft\",\n \"B\": \"18.5 psi/ft\",\n \"C\": \"33.8 psi/ft\",\n \"D\": \"35.2 psi/ft\",\n \"E\": \"47.6 psi/ft\",\n \"F\": \"60.1 psi/ft\",\n \"G\": \"55.5 psi/ft\",\n \"H\": \"25.4 psi/ft\",\n \"I\": \"30.0 psi/ft\",\n \"J\": \"50.7 psi/ft\"\n }\n}
input
question.json
{
"question": "A pseudoplastic-non-newtonian fluid is flowing through a schedule 40 standard steel pipe of 1(1/4) in. dia. at a rate of 15 gal/hr. Determine the pressure drop per foot of pipe. Properties of the fluid: density (\\rho) = 60lbm/ft^3; viscosity (\\mu_ap) = 22500 poise.",
"options": {
"A": "42.3 psi/ft",
"B": "18.5 psi/ft",
"C": "33.8 psi/ft",
"D": "35.2 psi/ft",
"E": "47.6 psi/ft",
"F": "60.1 psi/ft",
"G": "55.5 psi/ft",
"H": "25.4 psi/ft",
"I": "30.0 psi/ft",
"J": "50.7 psi/ft"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_162/ files.
Scored by a judge this pinned commit does not carry.
▸case_163Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A droplet of molten lead of average 1/8 india, at 750°F, falls froma height of 60 ft, and solidifies by the cooling effect of thesurrounding air. The surrounding temperature is 70°F. If thelead solidifies at 621°F, calculate the coefficient of heat transfer. For lead, C_p= 0.031 Btu/lbm-°F \\\\rho= 710lbm/ft^3 h_fusion= 10.6 Btu/lbm\",\n \"options\": {\n \"A\": \"59.2 Btu/hr-ft^2-°F\",\n \"B\": \"45.7 Btu/hr-ft^2-°F\",\n \"C\": \"52.8 Btu/hr-ft^2-°F\",\n \"D\": \"63.4 Btu/hr-ft^2-°F\"\n }\n}
input
question.json
{
"question": "A droplet of molten lead of average 1/8 india, at 750°F, falls froma height of 60 ft, and solidifies by the cooling effect of thesurrounding air. The surrounding temperature is 70°F. If thelead solidifies at 621°F, calculate the coefficient of heat transfer. For lead, C_p= 0.031 Btu/lbm-°F \\rho= 710lbm/ft^3 h_fusion= 10.6 Btu/lbm",
"options": {
"A": "59.2 Btu/hr-ft^2-°F",
"B": "45.7 Btu/hr-ft^2-°F",
"C": "52.8 Btu/hr-ft^2-°F",
"D": "63.4 Btu/hr-ft^2-°F"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_163/ files.
Scored by a judge this pinned commit does not carry.
▸case_164Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A farmer has a large field where he grows corn. He hires a crop-duster to spray the field, but the crop- duster mistakenly sprays a strong herbicide, rather than a pesticide, and kills all the corn plants in the field. Several days later, a construction company building a house near the farmer's field negligently hits a gas line and starts a fire that burns everything in the area, including the farmer's field. If the farmer brings negligence claims against the crop-duster and the construction company in order to recover damages for the destruction of his corn crop, the most likely result is\",\n \"options\": {\n \"A\": \"neither the crop-duster nor the construction company is liable, because neither is a \\\"but for\\\" cause of the harm.\",\n \"B\": \"both the crop-duster and the construction company are liable, because each engaged in a negligent action that was sufficient to destroy the farmer's corn crop.\",\n \"C\": \"only the crop-duster is liable, because its negligence occurred before the construction company's negligence.\",\n \"D\": \"both the crop-duster and the construction company are liable, because they both contributed to the loss of the farmer's corn crop.\",\n \"E\": \"neither the crop-duster nor the construction company is liable, because the farmer did not have insurance for his corn crop.\",\n \"F\": \"neither the crop-duster nor the construction company is liable, because the farmer failed to protect his field.\",\n \"G\": \"only the construction company is liable, because the fire caused more damage than the herbicide.\",\n \"H\": \"only the crop-duster is liable, because the construction company's fire was an unforeseeable event.\",\n \"I\": \"only the construction company is liable, because its negligence would have destroyed the farmer's corn crop no matter what the crop- duster did.\",\n \"J\": \"the crop-duster is liable for the value of the corn crop and the construction company is liable for the damage to the field.\"\n }\n}
input
question.json
{
"question": "A farmer has a large field where he grows corn. He hires a crop-duster to spray the field, but the crop- duster mistakenly sprays a strong herbicide, rather than a pesticide, and kills all the corn plants in the field. Several days later, a construction company building a house near the farmer's field negligently hits a gas line and starts a fire that burns everything in the area, including the farmer's field. If the farmer brings negligence claims against the crop-duster and the construction company in order to recover damages for the destruction of his corn crop, the most likely result is",
"options": {
"A": "neither the crop-duster nor the construction company is liable, because neither is a \"but for\" cause of the harm.",
"B": "both the crop-duster and the construction company are liable, because each engaged in a negligent action that was sufficient to destroy the farmer's corn crop.",
"C": "only the crop-duster is liable, because its negligence occurred before the construction company's negligence.",
"D": "both the crop-duster and the construction company are liable, because they both contributed to the loss of the farmer's corn crop.",
"E": "neither the crop-duster nor the construction company is liable, because the farmer did not have insurance for his corn crop.",
"F": "neither the crop-duster nor the construction company is liable, because the farmer failed to protect his field.",
"G": "only the construction company is liable, because the fire caused more damage than the herbicide.",
"H": "only the crop-duster is liable, because the construction company's fire was an unforeseeable event.",
"I": "only the construction company is liable, because its negligence would have destroyed the farmer's corn crop no matter what the crop- duster did.",
"J": "the crop-duster is liable for the value of the corn crop and the construction company is liable for the damage to the field."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_164/ files.
Scored by a judge this pinned commit does not carry.
▸case_165Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"An owner of a pizza and Italian foods restaurant opens a new location on a street where another owner has a burger restaurant. It seems like business is slow for the pizza place owner and he eyes up his competitor, the burger man, as the reason. The pizza owner starts to make false statements about the burger restaurant and its inferior ingredients, which he says have been causing ptomaine poisoning in some of the customers. When burger joint owner loses customers and business income to pizza place owner, can he sue for conversion?\",\n \"options\": {\n \"A\": \"No, because conversion consists of appropriating another's real property.\",\n \"B\": \"No, because the burger owner failed to prove that the pizza owner's statements were false.\",\n \"C\": \"No, because there was no interference with his ownership or right of possession to his personal property.\",\n \"D\": \"No, because the pizza owner has the right to advertise his business as he sees fit.\",\n \"E\": \"No, because the burger owner did not suffer any physical harm or damage to his property.\",\n \"F\": \"Yes, conversion is an intentional tort and the pizza owner acted intentionally to convert and did convert part of the burger business.\",\n \"G\": \"Yes, because the pizza owner committed a tort of defamation, which is a form of conversion.\",\n \"H\": \"Yes, because the pizza owner's actions caused economic harm to the burger owner.\",\n \"I\": \"Yes, because the pizza owner violated the burger owner's right to a fair competition.\",\n \"J\": \"Yes, because the pizza owner interfered with his right of peaceful enjoyment and disrupted his business activities.\"\n }\n}
input
question.json
{
"question": "An owner of a pizza and Italian foods restaurant opens a new location on a street where another owner has a burger restaurant. It seems like business is slow for the pizza place owner and he eyes up his competitor, the burger man, as the reason. The pizza owner starts to make false statements about the burger restaurant and its inferior ingredients, which he says have been causing ptomaine poisoning in some of the customers. When burger joint owner loses customers and business income to pizza place owner, can he sue for conversion?",
"options": {
"A": "No, because conversion consists of appropriating another's real property.",
"B": "No, because the burger owner failed to prove that the pizza owner's statements were false.",
"C": "No, because there was no interference with his ownership or right of possession to his personal property.",
"D": "No, because the pizza owner has the right to advertise his business as he sees fit.",
"E": "No, because the burger owner did not suffer any physical harm or damage to his property.",
"F": "Yes, conversion is an intentional tort and the pizza owner acted intentionally to convert and did convert part of the burger business.",
"G": "Yes, because the pizza owner committed a tort of defamation, which is a form of conversion.",
"H": "Yes, because the pizza owner's actions caused economic harm to the burger owner.",
"I": "Yes, because the pizza owner violated the burger owner's right to a fair competition.",
"J": "Yes, because the pizza owner interfered with his right of peaceful enjoyment and disrupted his business activities."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_165/ files.
Scored by a judge this pinned commit does not carry.
▸case_166Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Two men held-up a liquor store in a city. During the robbery, one of the participants shot and killed the owner of the store. Over a year later, police detectives arrested a defendant after obtaining reliable information that the defendant was the robber who was responsible for killing the store's owner. Afterward, the defendant was taken to the station house where he was booked for the crime and then incarcerated. The next day, the prosecuting attorney made a decision to delay indictment until he could discover the identity of the defendant's accomplice. The defendant was then released from police custody. In this jurisdiction the statute of limitations for murder is five years. Five months later, the prosecuting attorney, after unsuccessfully attempting to secure information on the defendant's cohort, indicted the defendant, charging him with felony murder. Shortly before trial, the only eyewitness to the crime died. He would have testified that the defendant did not participate in the robbery. The defendant's motion to dismiss the indictment because of the delay between the date of the crime and the date of the indictment will most probably be\",\n \"options\": {\n \"A\": \"denied, because the delay did not infringe upon the defendant's rights.\",\n \"B\": \"granted, because the delay in prosecuting the defendant actually prejudiced him.\",\n \"C\": \"granted, because the defendant was not given the opportunity to confront his accuser.\",\n \"D\": \"denied, because pre-indictment delay does not violate the Constitution, as long as it does not exceed an applicable statute of limitations.\",\n \"E\": \"denied, because the delay was not excessive and was for a permissible purpose.\",\n \"F\": \"granted, because the only eyewitness to the crime died before the trial.\",\n \"G\": \"denied, because the defendant was eventually charged within the statute of limitations.\",\n \"H\": \"denied, because the prosecutor had a valid reason to delay the indictment.\",\n \"I\": \"granted, because the prosecutor failed to find the defendant's accomplice.\",\n \"J\": \"granted, because the prosecutor is constitutionally required to press charges, if at all, within a reasonable time after probable cause is established.\"\n }\n}
input
question.json
{
"question": "Two men held-up a liquor store in a city. During the robbery, one of the participants shot and killed the owner of the store. Over a year later, police detectives arrested a defendant after obtaining reliable information that the defendant was the robber who was responsible for killing the store's owner. Afterward, the defendant was taken to the station house where he was booked for the crime and then incarcerated. The next day, the prosecuting attorney made a decision to delay indictment until he could discover the identity of the defendant's accomplice. The defendant was then released from police custody. In this jurisdiction the statute of limitations for murder is five years. Five months later, the prosecuting attorney, after unsuccessfully attempting to secure information on the defendant's cohort, indicted the defendant, charging him with felony murder. Shortly before trial, the only eyewitness to the crime died. He would have testified that the defendant did not participate in the robbery. The defendant's motion to dismiss the indictment because of the delay between the date of the crime and the date of the indictment will most probably be",
"options": {
"A": "denied, because the delay did not infringe upon the defendant's rights.",
"B": "granted, because the delay in prosecuting the defendant actually prejudiced him.",
"C": "granted, because the defendant was not given the opportunity to confront his accuser.",
"D": "denied, because pre-indictment delay does not violate the Constitution, as long as it does not exceed an applicable statute of limitations.",
"E": "denied, because the delay was not excessive and was for a permissible purpose.",
"F": "granted, because the only eyewitness to the crime died before the trial.",
"G": "denied, because the defendant was eventually charged within the statute of limitations.",
"H": "denied, because the prosecutor had a valid reason to delay the indictment.",
"I": "granted, because the prosecutor failed to find the defendant's accomplice.",
"J": "granted, because the prosecutor is constitutionally required to press charges, if at all, within a reasonable time after probable cause is established."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_166/ files.
Scored by a judge this pinned commit does not carry.
▸case_167Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"In a motor the armature resistance is 0.1 ohm. When connected across 110-volt mains the armature takes 20 amp, and its speed is 1,200 rpm. Determine its speed when the armature takes 50 amp from the same mains, with the field increased 10 percent.\",\n \"options\": {\n \"A\": \"850 rpm\",\n \"B\": \"1,320 rpm\",\n \"C\": \"1,000 rpm\",\n \"D\": \"1,061 rpm\",\n \"E\": \"1,150 rpm\",\n \"F\": \"1,100 rpm\",\n \"G\": \"1,080 rpm\",\n \"H\": \"1,250 rpm\",\n \"I\": \"900 rpm\",\n \"J\": \"1,200 rpm\"\n }\n}
input
question.json
{
"question": "In a motor the armature resistance is 0.1 ohm. When connected across 110-volt mains the armature takes 20 amp, and its speed is 1,200 rpm. Determine its speed when the armature takes 50 amp from the same mains, with the field increased 10 percent.",
"options": {
"A": "850 rpm",
"B": "1,320 rpm",
"C": "1,000 rpm",
"D": "1,061 rpm",
"E": "1,150 rpm",
"F": "1,100 rpm",
"G": "1,080 rpm",
"H": "1,250 rpm",
"I": "900 rpm",
"J": "1,200 rpm"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_167/ files.
Scored by a judge this pinned commit does not carry.
▸case_168Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"The statistical data for a certain community indicate that 25% of all girls are blondes, 50% of all girls and 75% of all blondes have blue eyes. If it is known that a girl has blue eyes, how much additional information is available if one is told that she is also a blonde?\",\n \"options\": {\n \"A\": \"0.85 bits\",\n \"B\": \"1 bit\",\n \"C\": \"2 bits\",\n \"D\": \"0.5 bits\",\n \"E\": \"1.25 bits\",\n \"F\": \"0.75 bits\",\n \"G\": \"0.25 bits\",\n \"H\": \"1.75 bits\",\n \"I\": \"1.415 bits\",\n \"J\": \"1.6 bits\"\n }\n}
input
question.json
{
"question": "The statistical data for a certain community indicate that 25% of all girls are blondes, 50% of all girls and 75% of all blondes have blue eyes. If it is known that a girl has blue eyes, how much additional information is available if one is told that she is also a blonde?",
"options": {
"A": "0.85 bits",
"B": "1 bit",
"C": "2 bits",
"D": "0.5 bits",
"E": "1.25 bits",
"F": "0.75 bits",
"G": "0.25 bits",
"H": "1.75 bits",
"I": "1.415 bits",
"J": "1.6 bits"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_168/ files.
Scored by a judge this pinned commit does not carry.
▸case_169Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A homeowner was holding his annual party at his home. His parties attracted a wide variety of diverse individuals. On the night in question, a defendant was loafing at the local bar when a number of the invited guests came in to purchase spirits for the party. The defendant decided to crash the party. He arrived at the homeowner's house around midnight when the party was in full swing. He went in through the front door, which was unlocked, and easily mingled with the other partygoers. During the party, the defendant wandered through the house and came upon a baseball, signed by a famous baseball player, in a glass case. Knowing the ball to be very valuable, the defendant put it under his jacket and made his way down the stairs and out the front door. If the defendant is subsequently charged with burglary, he will probably be found\",\n \"options\": {\n \"A\": \"guilty, because he stole a valuable item.\",\n \"B\": \"not guilty, because he did not use force to enter the house.\",\n \"C\": \"not guilty, because the front door was unlocked.\",\n \"D\": \"guilty, because he intended to steal the baseball.\",\n \"E\": \"guilty, because he didn't have an invitation to the party.\",\n \"F\": \"not guilty, because he crashed the party only to have a good time.\",\n \"G\": \"guilty, because he crashed the party.\",\n \"H\": \"not guilty, because he left the party without causing any harm.\",\n \"I\": \"guilty, because he had the intention to commit a crime inside the house.\",\n \"J\": \"not guilty, because he did not break into the house.\"\n }\n}
input
question.json
{
"question": "A homeowner was holding his annual party at his home. His parties attracted a wide variety of diverse individuals. On the night in question, a defendant was loafing at the local bar when a number of the invited guests came in to purchase spirits for the party. The defendant decided to crash the party. He arrived at the homeowner's house around midnight when the party was in full swing. He went in through the front door, which was unlocked, and easily mingled with the other partygoers. During the party, the defendant wandered through the house and came upon a baseball, signed by a famous baseball player, in a glass case. Knowing the ball to be very valuable, the defendant put it under his jacket and made his way down the stairs and out the front door. If the defendant is subsequently charged with burglary, he will probably be found",
"options": {
"A": "guilty, because he stole a valuable item.",
"B": "not guilty, because he did not use force to enter the house.",
"C": "not guilty, because the front door was unlocked.",
"D": "guilty, because he intended to steal the baseball.",
"E": "guilty, because he didn't have an invitation to the party.",
"F": "not guilty, because he crashed the party only to have a good time.",
"G": "guilty, because he crashed the party.",
"H": "not guilty, because he left the party without causing any harm.",
"I": "guilty, because he had the intention to commit a crime inside the house.",
"J": "not guilty, because he did not break into the house."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_169/ files.
Scored by a judge this pinned commit does not carry.
▸case_170Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A polished stainless steel surface is electrically heated from the bottom to boil water at atmospheric pressure. If the surface temperature is 230 F, calculate the heat flux from the surface to the water and compare this value with the critical heat flux of nucleate boiling.\",\n \"options\": {\n \"A\": \"15000 Btu/hr-ft^2\",\n \"B\": \"30000 Btu/hr-ft^2\",\n \"C\": \"22000 Btu/hr-ft^2\",\n \"D\": \"5000 Btu/hr-ft^2\",\n \"E\": \"16000 Btu/hr-ft^2\",\n \"F\": \"12500 Btu/hr-ft^2\",\n \"G\": \"9500 Btu/hr-ft^2\",\n \"H\": \"10888.25 Btu/hr-ft^2\",\n \"I\": \"7000 Btu/hr-ft^2\",\n \"J\": \"20000 Btu/hr-ft^2\"\n }\n}
input
question.json
{
"question": "A polished stainless steel surface is electrically heated from the bottom to boil water at atmospheric pressure. If the surface temperature is 230 F, calculate the heat flux from the surface to the water and compare this value with the critical heat flux of nucleate boiling.",
"options": {
"A": "15000 Btu/hr-ft^2",
"B": "30000 Btu/hr-ft^2",
"C": "22000 Btu/hr-ft^2",
"D": "5000 Btu/hr-ft^2",
"E": "16000 Btu/hr-ft^2",
"F": "12500 Btu/hr-ft^2",
"G": "9500 Btu/hr-ft^2",
"H": "10888.25 Btu/hr-ft^2",
"I": "7000 Btu/hr-ft^2",
"J": "20000 Btu/hr-ft^2"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_170/ files.
Scored by a judge this pinned commit does not carry.
▸case_171Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Estimate the depth to which a water pipe line is to be laid suchthat the water flowing in the pipe does not freeze due to lowtemperatures at the surface. The mean temperature duringcold season is 5°C and minimum temperature goes downto - 10°C and exists for a maximum period of 48 hours. Assume the properties of the earth as k = 0.52 W/m - °K,\\\\rho = 1840 kg/m^3 C_p = 2050 J/kg - °K. Table 1: The Complimentary Error Function erfc (\\\\eta) = 1- {2 / \\\\surd\\\\pi}^\\\\eta\\\\int_oe^-(u)2 du \\\\eta erfc (\\\\eta) \\\\eta erfc (\\\\eta) 0.0 1.0000 1.1 0.11980 0.05 0.9436 1.2 0.08969 0.1 0.8875 1.3 0.06599 0.15 0.8320 1.4 0.04772 0.2 0.7773 1.5 0.03390 0.2S 0.7237 1.6 0.02365 0.3 0.6714 1.7 0.01621 0.35 0.6206 1.8 0.01091 0.4 0.5716 1.9 0.00721 0.45 0.5245 2.0 0.00468 0.5 0.4795 2.1 0.00298 0.55 0.4367 2.2 0.00186 0.6 0.3961 2.3 0.001143 0.65 0.3580 2.4 0.000689 0.7 0.3222 2.5 0.000407 0.75 0.2889 2.6 0.000236 0.8 0.2579 2.7 0.000134 0.85 0.2293 2.8 0.000075 0.9 0.2031 2.9 0.000041 0.95 0.1791 3.0 0.000022 1.00 0.1573\",\n \"options\": {\n \"A\": \"25 cm\",\n \"B\": \"35 cm\",\n \"C\": \"15 cm\",\n \"D\": \"50 cm\",\n \"E\": \"5 cm\",\n \"F\": \"45 cm\",\n \"G\": \"10 cm\",\n \"H\": \"20 cm\",\n \"I\": \"40 cm\",\n \"J\": \"30 cm\"\n }\n}
input
question.json
{
"question": "Estimate the depth to which a water pipe line is to be laid suchthat the water flowing in the pipe does not freeze due to lowtemperatures at the surface. The mean temperature duringcold season is 5°C and minimum temperature goes downto - 10°C and exists for a maximum period of 48 hours. Assume the properties of the earth as k = 0.52 W/m - °K,\\rho = 1840 kg/m^3 C_p = 2050 J/kg - °K. Table 1: The Complimentary Error Function erfc (\\eta) = 1- {2 / \\surd\\pi}^\\eta\\int_oe^-(u)2 du \\eta erfc (\\eta) \\eta erfc (\\eta) 0.0 1.0000 1.1 0.11980 0.05 0.9436 1.2 0.08969 0.1 0.8875 1.3 0.06599 0.15 0.8320 1.4 0.04772 0.2 0.7773 1.5 0.03390 0.2S 0.7237 1.6 0.02365 0.3 0.6714 1.7 0.01621 0.35 0.6206 1.8 0.01091 0.4 0.5716 1.9 0.00721 0.45 0.5245 2.0 0.00468 0.5 0.4795 2.1 0.00298 0.55 0.4367 2.2 0.00186 0.6 0.3961 2.3 0.001143 0.65 0.3580 2.4 0.000689 0.7 0.3222 2.5 0.000407 0.75 0.2889 2.6 0.000236 0.8 0.2579 2.7 0.000134 0.85 0.2293 2.8 0.000075 0.9 0.2031 2.9 0.000041 0.95 0.1791 3.0 0.000022 1.00 0.1573",
"options": {
"A": "25 cm",
"B": "35 cm",
"C": "15 cm",
"D": "50 cm",
"E": "5 cm",
"F": "45 cm",
"G": "10 cm",
"H": "20 cm",
"I": "40 cm",
"J": "30 cm"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_171/ files.
Scored by a judge this pinned commit does not carry.
▸case_172Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A senior associate in a law firm was conducting spot checks of other associates as they left the office to determine who had taken some documents that were missing from the firm's library. A young associate was leaving his office for the day when he was stopped by the defendant, who asked him to open his briefcase. The senior associate explained that some files from the law office had been stolen. Startled, the young associate refused to permit the senior associate to look in his briefcase. The young associate said that he had personal papers in his briefcase and he didn't want the senior associate to see them. Having his suspicions aroused, the senior associate raised his clenched fist and exclaimed, \\\"If you don't hand over that briefcase, I'm going to punch you in the mouth. \\\" The young associate, who, unknown to the senior associate, suffered from a serious heart condition, had a seizure and died as a result of fright produced by the threatened attack. Which of the following is the most serious crime for which the senior associate should be found guilty?\",\n \"options\": {\n \"A\": \"Voluntary manslaughtçr.\",\n \"B\": \"No crime, it was self-defense.\",\n \"C\": \"Aggravated assault.\",\n \"D\": \"Involuntary manslaughter.\",\n \"E\": \"Assault.\",\n \"F\": \"Attempted murder.\",\n \"G\": \"Murder.\",\n \"H\": \"Negligent homicide.\",\n \"I\": \"Battery.\",\n \"J\": \"Criminal Threat.\"\n }\n}
input
question.json
{
"question": "A senior associate in a law firm was conducting spot checks of other associates as they left the office to determine who had taken some documents that were missing from the firm's library. A young associate was leaving his office for the day when he was stopped by the defendant, who asked him to open his briefcase. The senior associate explained that some files from the law office had been stolen. Startled, the young associate refused to permit the senior associate to look in his briefcase. The young associate said that he had personal papers in his briefcase and he didn't want the senior associate to see them. Having his suspicions aroused, the senior associate raised his clenched fist and exclaimed, \"If you don't hand over that briefcase, I'm going to punch you in the mouth. \" The young associate, who, unknown to the senior associate, suffered from a serious heart condition, had a seizure and died as a result of fright produced by the threatened attack. Which of the following is the most serious crime for which the senior associate should be found guilty?",
"options": {
"A": "Voluntary manslaughtçr.",
"B": "No crime, it was self-defense.",
"C": "Aggravated assault.",
"D": "Involuntary manslaughter.",
"E": "Assault.",
"F": "Attempted murder.",
"G": "Murder.",
"H": "Negligent homicide.",
"I": "Battery.",
"J": "Criminal Threat."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_172/ files.
Scored by a judge this pinned commit does not carry.
▸case_173Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A penny is to be tossed 3 times. What is the probability there will be 2 heads and 1 tail?\",\n \"options\": {\n \"A\": \"3/4\",\n \"B\": \"1/4\",\n \"C\": \"1/3\",\n \"D\": \"1/8\",\n \"E\": \"1/16\",\n \"F\": \"1/2\",\n \"G\": \"7/8\",\n \"H\": \"3/8\",\n \"I\": \"2/3\",\n \"J\": \"5/8\"\n }\n}
input
question.json
{
"question": "A penny is to be tossed 3 times. What is the probability there will be 2 heads and 1 tail?",
"options": {
"A": "3/4",
"B": "1/4",
"C": "1/3",
"D": "1/8",
"E": "1/16",
"F": "1/2",
"G": "7/8",
"H": "3/8",
"I": "2/3",
"J": "5/8"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_173/ files.
Scored by a judge this pinned commit does not carry.
▸case_174Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A multiple disc clutch is used to operate the speed regulating gears of a lathe. If the torque delivered by the clutch is 400 lbs.-in., evaluate the number of discs necessary and the force acting on them while the clutch is in operation. Assume a factor of safety of 2 and limit the disc friction surface sizes between 5 in. and 3 in.\",\n \"options\": {\n \"A\": \"Three discs, Force on the discs = 1028 lbs.\",\n \"B\": \"Four discs, Force on the discs = 1200 lbs.\",\n \"C\": \"Two discs, Force on the discs = 1600 lbs.\",\n \"D\": \"Four discs, Force on the discs = 800 lbs.\",\n \"E\": \"Six discs, Force on the discs = 666.7 lbs.\",\n \"F\": \"Two discs, Force on the discs = 1256.6 lbs.\",\n \"G\": \"One disc, Force on the discs = 2000 lbs.\",\n \"H\": \"Seven discs, Force on the discs = 571.4 lbs.\",\n \"I\": \"Five discs, Force on the discs = 1600 lbs.\",\n \"J\": \"Eight discs, Force on the discs = 500 lbs.\"\n }\n}
input
question.json
{
"question": "A multiple disc clutch is used to operate the speed regulating gears of a lathe. If the torque delivered by the clutch is 400 lbs.-in., evaluate the number of discs necessary and the force acting on them while the clutch is in operation. Assume a factor of safety of 2 and limit the disc friction surface sizes between 5 in. and 3 in.",
"options": {
"A": "Three discs, Force on the discs = 1028 lbs.",
"B": "Four discs, Force on the discs = 1200 lbs.",
"C": "Two discs, Force on the discs = 1600 lbs.",
"D": "Four discs, Force on the discs = 800 lbs.",
"E": "Six discs, Force on the discs = 666.7 lbs.",
"F": "Two discs, Force on the discs = 1256.6 lbs.",
"G": "One disc, Force on the discs = 2000 lbs.",
"H": "Seven discs, Force on the discs = 571.4 lbs.",
"I": "Five discs, Force on the discs = 1600 lbs.",
"J": "Eight discs, Force on the discs = 500 lbs."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_174/ files.
Scored by a judge this pinned commit does not carry.
▸case_175Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Find the equilibrium constant for the reaction CO + H_2O \\\\rightarrow CO_2 + H_2 at77°F. Assume ideal gas behavior.\",\n \"options\": {\n \"A\": \"120,000\",\n \"B\": \"65,000\",\n \"C\": \"85,000\",\n \"D\": \"130,000\",\n \"E\": \"98,000\",\n \"F\": \"55,000\",\n \"G\": \"45,000\",\n \"H\": \"90,000\",\n \"I\": \"75,000\",\n \"J\": \"110,000\"\n }\n}
input
question.json
{
"question": "Find the equilibrium constant for the reaction CO + H_2O \\rightarrow CO_2 + H_2 at77°F. Assume ideal gas behavior.",
"options": {
"A": "120,000",
"B": "65,000",
"C": "85,000",
"D": "130,000",
"E": "98,000",
"F": "55,000",
"G": "45,000",
"H": "90,000",
"I": "75,000",
"J": "110,000"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_175/ files.
Scored by a judge this pinned commit does not carry.
▸case_176Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Calculate the full-load torque, in pound-feet and ounce-inches, of a 1/50-hp 1,500-rpm shaded-pole motor. A 1/20-hp 1,550-rpm shaded-pole motor has a maximum torqueof 38 oz-in. and a starting torque of 12 oz-in. What percentagesare these torques with respect to the full-load torque?\",\n \"options\": {\n \"A\": \"0.07 lb-ft, 13.4 oz-in, 118%, 37%\",\n \"B\": \"0.07 lb-ft, 12 oz-in, 119%, 39%\",\n \"C\": \"0.08 lb-ft, 14 oz-in, 115%, 35%\",\n \"D\": \"0.10 lb-ft, 16 oz-in, 112%, 33%\",\n \"E\": \"0.09 lb-ft, 15 oz-in, 110%, 34%\",\n \"F\": \"0.05 lb-ft, 10 oz-in, 120%, 40%\",\n \"G\": \"0.05 lb-ft, 9 oz-in, 122%, 41%\",\n \"H\": \"0.06 lb-ft, 11 oz-in, 116%, 36%\",\n \"I\": \"0.04 lb-ft, 8 oz-in, 125%, 42%\",\n \"J\": \"0.06 lb-ft, 12 oz-in, 119%, 38%\"\n }\n}
input
question.json
{
"question": "Calculate the full-load torque, in pound-feet and ounce-inches, of a 1/50-hp 1,500-rpm shaded-pole motor. A 1/20-hp 1,550-rpm shaded-pole motor has a maximum torqueof 38 oz-in. and a starting torque of 12 oz-in. What percentagesare these torques with respect to the full-load torque?",
"options": {
"A": "0.07 lb-ft, 13.4 oz-in, 118%, 37%",
"B": "0.07 lb-ft, 12 oz-in, 119%, 39%",
"C": "0.08 lb-ft, 14 oz-in, 115%, 35%",
"D": "0.10 lb-ft, 16 oz-in, 112%, 33%",
"E": "0.09 lb-ft, 15 oz-in, 110%, 34%",
"F": "0.05 lb-ft, 10 oz-in, 120%, 40%",
"G": "0.05 lb-ft, 9 oz-in, 122%, 41%",
"H": "0.06 lb-ft, 11 oz-in, 116%, 36%",
"I": "0.04 lb-ft, 8 oz-in, 125%, 42%",
"J": "0.06 lb-ft, 12 oz-in, 119%, 38%"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_176/ files.
Scored by a judge this pinned commit does not carry.
▸case_177Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Which of the following will not terminate a contract by operation of law?\",\n \"options\": {\n \"A\": \"Change in law that makes contract impossible\",\n \"B\": \"Bankruptcy of one party\",\n \"C\": \"Expiration of the contract term\",\n \"D\": \"Partial performance\",\n \"E\": \"Supervening illegality\",\n \"F\": \"Death or insanity of a party.\",\n \"G\": \"Fulfillment of contract\",\n \"H\": \"Destruction of the subject matter of contract.\"\n }\n}
input
question.json
{
"question": "Which of the following will not terminate a contract by operation of law?",
"options": {
"A": "Change in law that makes contract impossible",
"B": "Bankruptcy of one party",
"C": "Expiration of the contract term",
"D": "Partial performance",
"E": "Supervening illegality",
"F": "Death or insanity of a party.",
"G": "Fulfillment of contract",
"H": "Destruction of the subject matter of contract."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_177/ files.
Scored by a judge this pinned commit does not carry.
▸case_178Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Maine's famous aphorism that 'the movement of progressive societies has hitherto been a movement from Status to Contract' is often misunderstood. In what way?\",\n \"options\": {\n \"A\": \"His concept of status is misrepresented.\",\n \"B\": \"It is wrongly considered to be a universal rule.\",\n \"C\": \"It is misinterpreted as a prediction.\",\n \"D\": \"His idea is considered inapplicable to Western legal systems.\",\n \"E\": \"It is incorrectly related to economic progression.\",\n \"F\": \"It is taken literally.\",\n \"G\": \"The historical context of the aphorism is often overlooked.\",\n \"H\": \"His use of the word 'progressive' is often misconstrued.\",\n \"I\": \"His concept of 'contract' is often misinterpreted.\",\n \"J\": \"It is mistakenly thought to apply to all types of societies.\"\n }\n}
input
question.json
{
"question": "Maine's famous aphorism that 'the movement of progressive societies has hitherto been a movement from Status to Contract' is often misunderstood. In what way?",
"options": {
"A": "His concept of status is misrepresented.",
"B": "It is wrongly considered to be a universal rule.",
"C": "It is misinterpreted as a prediction.",
"D": "His idea is considered inapplicable to Western legal systems.",
"E": "It is incorrectly related to economic progression.",
"F": "It is taken literally.",
"G": "The historical context of the aphorism is often overlooked.",
"H": "His use of the word 'progressive' is often misconstrued.",
"I": "His concept of 'contract' is often misinterpreted.",
"J": "It is mistakenly thought to apply to all types of societies."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_178/ files.
Scored by a judge this pinned commit does not carry.
▸case_179Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A man was prosecuted for assault and battery after he admitted striking a victim with a pool cue during a barroom argument. The man claimed that he acted in self-defense after he was attacked by the victim, who was drunk and belligerent. As his first defense witness, the man calls his neighbor to testify that the man is a good neighbor. The neighbor's testimony is\",\n \"options\": {\n \"A\": \"admissible, because it could sway the jury's opinion in favor of the man.\",\n \"B\": \"inadmissible, because the neighbor's opinion is not considered reliable evidence.\",\n \"C\": \"admissible, because it is relevant to show the improbability of the man's having committed an unprovoked attack.\",\n \"D\": \"inadmissible, because it is merely the opinion of a character witness.\",\n \"E\": \"admissible, because it could help establish the man's reputation.\",\n \"F\": \"admissible, because it is relevant to support the man's credibility.\",\n \"G\": \"inadmissible, because it is not directed toward a pertinent trait of the man's character.\",\n \"H\": \"admissible, because it shows the man's peaceful nature.\",\n \"I\": \"inadmissible, because it does not provide direct evidence of the man's actions during the incident.\"\n }\n}
input
question.json
{
"question": "A man was prosecuted for assault and battery after he admitted striking a victim with a pool cue during a barroom argument. The man claimed that he acted in self-defense after he was attacked by the victim, who was drunk and belligerent. As his first defense witness, the man calls his neighbor to testify that the man is a good neighbor. The neighbor's testimony is",
"options": {
"A": "admissible, because it could sway the jury's opinion in favor of the man.",
"B": "inadmissible, because the neighbor's opinion is not considered reliable evidence.",
"C": "admissible, because it is relevant to show the improbability of the man's having committed an unprovoked attack.",
"D": "inadmissible, because it is merely the opinion of a character witness.",
"E": "admissible, because it could help establish the man's reputation.",
"F": "admissible, because it is relevant to support the man's credibility.",
"G": "inadmissible, because it is not directed toward a pertinent trait of the man's character.",
"H": "admissible, because it shows the man's peaceful nature.",
"I": "inadmissible, because it does not provide direct evidence of the man's actions during the incident."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_179/ files.
Scored by a judge this pinned commit does not carry.
▸case_180Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A defendant decided to rob a bar because he knew that the bar was now utilizing a new security system that was faulty and that nighttime was the best time to break in. Late one night, the defendant broke into the bar and proceeded to load large kegs of beer onto his truck. As he was leaving, the defendant inadvertently threw his cigarette butt into a trash can behind the bar. Soon afterward, the trash ignited into a small fire. Although the defendant had time to douse the fire without any danger to himself, he did not do so. Instead, he climbed into the truck and drove away. Although the fire spread beyond the trashcan, the fire department responded quickly and was able to put it out. The floor behind the bar was charred. The statutes for both burglary and arson in this jurisdiction include residential and business structures. The defendant should be convicted of which of the following crimes?\",\n \"options\": {\n \"A\": \"Robbery and burglary.\",\n \"B\": \"Only Arson.\",\n \"C\": \"Only burglary.\",\n \"D\": \"Arson and larceny.\",\n \"E\": \"Arson, larceny, and robbery.\",\n \"F\": \"Larceny and burglary.\",\n \"G\": \"Arson, robbery, burglary, and larceny.\",\n \"H\": \"Arson, burglary, and larceny.\",\n \"I\": \"Arson and burglary.\",\n \"J\": \"Arson, robbery, and burglary.\"\n }\n}
input
question.json
{
"question": "A defendant decided to rob a bar because he knew that the bar was now utilizing a new security system that was faulty and that nighttime was the best time to break in. Late one night, the defendant broke into the bar and proceeded to load large kegs of beer onto his truck. As he was leaving, the defendant inadvertently threw his cigarette butt into a trash can behind the bar. Soon afterward, the trash ignited into a small fire. Although the defendant had time to douse the fire without any danger to himself, he did not do so. Instead, he climbed into the truck and drove away. Although the fire spread beyond the trashcan, the fire department responded quickly and was able to put it out. The floor behind the bar was charred. The statutes for both burglary and arson in this jurisdiction include residential and business structures. The defendant should be convicted of which of the following crimes?",
"options": {
"A": "Robbery and burglary.",
"B": "Only Arson.",
"C": "Only burglary.",
"D": "Arson and larceny.",
"E": "Arson, larceny, and robbery.",
"F": "Larceny and burglary.",
"G": "Arson, robbery, burglary, and larceny.",
"H": "Arson, burglary, and larceny.",
"I": "Arson and burglary.",
"J": "Arson, robbery, and burglary."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_180/ files.
Scored by a judge this pinned commit does not carry.
▸case_181Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"The VSWR (voltage standing wave ratio) on a 50-Ohm (characteristic impedance) transmission line is 2. The distance between successive voltage minima is 40 cm while the distance from the load to the first minima is 10 cm. What is the reflection coefficient and load impedance?\",\n \"options\": {\n \"A\": \"\\\\Gamma_L = [(- j) / 3], Z_L = 50 - 30j ohm\",\n \"B\": \"\\\\Gamma_L = [(- j) / 3], Z_L = 60 - 30j ohm\",\n \"C\": \"\\\\Gamma_L = [(- j) / 6], Z_L = 50 - 10j ohm\",\n \"D\": \"\\\\Gamma_L = [(- j) / 5], Z_L = 30 - 20j ohm\",\n \"E\": \"\\\\Gamma_L = [(- j) / 3], Z_L = 40 - 40j ohm\",\n \"F\": \"\\\\Gamma_L = [(+ j) / 2], Z_L = 60 + 60j ohm\",\n \"G\": \"\\\\Gamma_L = [(+ j) / 3], Z_L = 60 + 30j ohm\",\n \"H\": \"\\\\Gamma_L = [(- j) / 4], Z_L = 50 - 40j ohm\",\n \"I\": \"\\\\Gamma_L = [(- j) / 3], Z_L = 40 - 30j ohm\",\n \"J\": \"\\\\Gamma_L = [(- j) / 2], Z_L = 40 - 30j ohm\"\n }\n}
input
question.json
{
"question": "The VSWR (voltage standing wave ratio) on a 50-Ohm (characteristic impedance) transmission line is 2. The distance between successive voltage minima is 40 cm while the distance from the load to the first minima is 10 cm. What is the reflection coefficient and load impedance?",
"options": {
"A": "\\Gamma_L = [(- j) / 3], Z_L = 50 - 30j ohm",
"B": "\\Gamma_L = [(- j) / 3], Z_L = 60 - 30j ohm",
"C": "\\Gamma_L = [(- j) / 6], Z_L = 50 - 10j ohm",
"D": "\\Gamma_L = [(- j) / 5], Z_L = 30 - 20j ohm",
"E": "\\Gamma_L = [(- j) / 3], Z_L = 40 - 40j ohm",
"F": "\\Gamma_L = [(+ j) / 2], Z_L = 60 + 60j ohm",
"G": "\\Gamma_L = [(+ j) / 3], Z_L = 60 + 30j ohm",
"H": "\\Gamma_L = [(- j) / 4], Z_L = 50 - 40j ohm",
"I": "\\Gamma_L = [(- j) / 3], Z_L = 40 - 30j ohm",
"J": "\\Gamma_L = [(- j) / 2], Z_L = 40 - 30j ohm"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_181/ files.
Scored by a judge this pinned commit does not carry.
▸case_182Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"The nonlinear characteristic of a capacitor is given by q(t) = 0.5\\\\nu^2 (t) The voltage across this capacitor is given by \\\\nu(t) = 1 + 0.5 sin t Find the current through this capacitor.\",\n \"options\": {\n \"A\": \"i(t) = 0.5cos t + 0.25 sin 2t\",\n \"B\": \"i(t) = .5cos 2t + .125 sin 2t\",\n \"C\": \"i(t) = 0.5cos 2t + 0.125 cos t\",\n \"D\": \"i(t) = .5sin + .125 cos 2t\",\n \"E\": \"i(t) = 1.0cos t + 0.125 sin 2t\",\n \"F\": \"i(t) = .5cost + .125 sin t\",\n \"G\": \"i(t) = .5cost + .125 sin 2t\",\n \"H\": \"i(t) = ν(t)sin t + 0.125 cos t\",\n \"I\": \"i(t) = ν(t)cos t + 0.25 sin t\",\n \"J\": \"i(t) = 0.5sin t + 0.125 sin t\"\n }\n}
input
question.json
{
"question": "The nonlinear characteristic of a capacitor is given by q(t) = 0.5\\nu^2 (t) The voltage across this capacitor is given by \\nu(t) = 1 + 0.5 sin t Find the current through this capacitor.",
"options": {
"A": "i(t) = 0.5cos t + 0.25 sin 2t",
"B": "i(t) = .5cos 2t + .125 sin 2t",
"C": "i(t) = 0.5cos 2t + 0.125 cos t",
"D": "i(t) = .5sin + .125 cos 2t",
"E": "i(t) = 1.0cos t + 0.125 sin 2t",
"F": "i(t) = .5cost + .125 sin t",
"G": "i(t) = .5cost + .125 sin 2t",
"H": "i(t) = ν(t)sin t + 0.125 cos t",
"I": "i(t) = ν(t)cos t + 0.25 sin t",
"J": "i(t) = 0.5sin t + 0.125 sin t"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_182/ files.
Scored by a judge this pinned commit does not carry.
▸case_183Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A man told his neighbor, \\\"If you will mow my lawn, I will pay you $50. \\\" The neighbor then purchased gasoline for the mower so he could mow the lawn. The neighbor wheeled his lawn mower to the man's property and started the engine. At that moment, the man suddenly yelled, \\\"I hereby revoke my offer. \\\"If the neighbor thereafter mows the man's lawn, will he recover the $50?\",\n \"options\": {\n \"A\": \"Yes, because there was an offer for a unilateral contract that became irrevocable prior to the man's attempted revocation.\",\n \"B\": \"No, because the neighbor had done nothing more than prepare to accept the offer prior to the man's revocation.\",\n \"C\": \"Yes, because the neighbor can sue for breach of oral contract.\",\n \"D\": \"Yes, because the neighbor had begun the task before the man revoked his offer.\",\n \"E\": \"Yes, under the doctrine of quasi-contract.\",\n \"F\": \"No, because there was no written contract and hence no legal obligation for the man to pay.\",\n \"G\": \"No, because the neighbor did not get a written agreement from the man.\",\n \"H\": \"No, because the man's revocation was effective, since the neighbor had not completed performance.\",\n \"I\": \"No, because the man revoked his offer before the task was completed.\",\n \"J\": \"Yes, because the neighbor had already incurred expenses in preparation to fulfill the contract.\"\n }\n}
input
question.json
{
"question": "A man told his neighbor, \"If you will mow my lawn, I will pay you $50. \" The neighbor then purchased gasoline for the mower so he could mow the lawn. The neighbor wheeled his lawn mower to the man's property and started the engine. At that moment, the man suddenly yelled, \"I hereby revoke my offer. \"If the neighbor thereafter mows the man's lawn, will he recover the $50?",
"options": {
"A": "Yes, because there was an offer for a unilateral contract that became irrevocable prior to the man's attempted revocation.",
"B": "No, because the neighbor had done nothing more than prepare to accept the offer prior to the man's revocation.",
"C": "Yes, because the neighbor can sue for breach of oral contract.",
"D": "Yes, because the neighbor had begun the task before the man revoked his offer.",
"E": "Yes, under the doctrine of quasi-contract.",
"F": "No, because there was no written contract and hence no legal obligation for the man to pay.",
"G": "No, because the neighbor did not get a written agreement from the man.",
"H": "No, because the man's revocation was effective, since the neighbor had not completed performance.",
"I": "No, because the man revoked his offer before the task was completed.",
"J": "Yes, because the neighbor had already incurred expenses in preparation to fulfill the contract."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_183/ files.
Scored by a judge this pinned commit does not carry.
▸case_184Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"The President and sole stockholder of a small business negotiated to sell the business to an individual investor. The parties met and signed a \\\"Letter Agreement\\\", which set forth the price, down payment amount, financing terms, the assets to be included, and a host of other pertinent details. The final closing date was not stated, but the purchasers left a down payment of $20,000 with the seller. The letter stated that the terms comprised their essential agreement, but that a formal document would be drafted by the parties' attorneys. Before that happened, the seller returned the check and refused to perform. The investor brought an equity action for specific performance to compel the sale. The equity court ruled that there was a complete and binding contract that must be specifically performed. Will the appellate court likely affirm the lower court's decision?\",\n \"options\": {\n \"A\": \"Yes, because the seller accepted the down payment, indicating acceptance of the agreement.\",\n \"B\": \"No, because this type of letter is basically a \\\"gentlemen's agreement\\\" that is not intended to be legally binding.\",\n \"C\": \"No, because the seller has the right to refuse the sale at any time before the final contract is signed.\",\n \"D\": \"Yes, because a letter of intent is legally binding if it sets forth the price, describes the business and is signed by both parties.\",\n \"E\": \"No, because the terms of the agreement were not finalized by the parties' attorneys.\",\n \"F\": \"No, because a letter agreement is not a legally binding document.\",\n \"G\": \"Yes, because the letter agreement contained all the necessary details for the sale.\",\n \"H\": \"No, because closing date was left out and the attorneys were supposed to compose the final contract with all of the remaining details put in.\",\n \"I\": \"Yes, because the buyer left a down payment, signifying their intention to follow through with the purchase.\",\n \"J\": \"Yes, because where the letter contains all of the essential terms required for a contract, it will be enforced as a contract.\"\n }\n}
input
question.json
{
"question": "The President and sole stockholder of a small business negotiated to sell the business to an individual investor. The parties met and signed a \"Letter Agreement\", which set forth the price, down payment amount, financing terms, the assets to be included, and a host of other pertinent details. The final closing date was not stated, but the purchasers left a down payment of $20,000 with the seller. The letter stated that the terms comprised their essential agreement, but that a formal document would be drafted by the parties' attorneys. Before that happened, the seller returned the check and refused to perform. The investor brought an equity action for specific performance to compel the sale. The equity court ruled that there was a complete and binding contract that must be specifically performed. Will the appellate court likely affirm the lower court's decision?",
"options": {
"A": "Yes, because the seller accepted the down payment, indicating acceptance of the agreement.",
"B": "No, because this type of letter is basically a \"gentlemen's agreement\" that is not intended to be legally binding.",
"C": "No, because the seller has the right to refuse the sale at any time before the final contract is signed.",
"D": "Yes, because a letter of intent is legally binding if it sets forth the price, describes the business and is signed by both parties.",
"E": "No, because the terms of the agreement were not finalized by the parties' attorneys.",
"F": "No, because a letter agreement is not a legally binding document.",
"G": "Yes, because the letter agreement contained all the necessary details for the sale.",
"H": "No, because closing date was left out and the attorneys were supposed to compose the final contract with all of the remaining details put in.",
"I": "Yes, because the buyer left a down payment, signifying their intention to follow through with the purchase.",
"J": "Yes, because where the letter contains all of the essential terms required for a contract, it will be enforced as a contract."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_184/ files.
Scored by a judge this pinned commit does not carry.
▸case_185Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A solid cylindrical shaft, 2 in. dia. 18 in. long, is used for the transmission of mechanical power at a rate of 45 hp running at 1500 rpm. The modulus of rigidity for the material used is 14 × 10^6 psi. Calculate a) the shear stress acting on the shaft; b) the torsional deflection of the shaft.\",\n \"options\": {\n \"A\": \"Shear stress is approximately 1300 psi and torsional deflection is approximately 0.095 degrees\",\n \"B\": \"Shear stress is approximately 1500 psi and torsional deflection is approximately 0.110 degrees\",\n \"C\": \"Shear stress is approximately 1204 psi and torsional deflection is approximately 0.089 degrees\",\n \"D\": \"Shear stress is approximately 950 psi and torsional deflection is approximately 0.075 degrees\",\n \"E\": \"Shear stress is approximately 800 psi and torsional deflection is approximately 0.065 degrees\",\n \"F\": \"Shear stress is approximately 1000 psi and torsional deflection is approximately 0.080 degrees\",\n \"G\": \"Shear stress is approximately 1400 psi and torsional deflection is approximately 0.105 degrees\",\n \"H\": \"Shear stress is approximately 900 psi and torsional deflection is approximately 0.070 degrees\",\n \"I\": \"Shear stress is approximately 1600 psi and torsional deflection is approximately 0.120 degrees\",\n \"J\": \"Shear stress is approximately 1100 psi and torsional deflection is approximately 0.100 degrees\"\n }\n}
input
question.json
{
"question": "A solid cylindrical shaft, 2 in. dia. 18 in. long, is used for the transmission of mechanical power at a rate of 45 hp running at 1500 rpm. The modulus of rigidity for the material used is 14 × 10^6 psi. Calculate a) the shear stress acting on the shaft; b) the torsional deflection of the shaft.",
"options": {
"A": "Shear stress is approximately 1300 psi and torsional deflection is approximately 0.095 degrees",
"B": "Shear stress is approximately 1500 psi and torsional deflection is approximately 0.110 degrees",
"C": "Shear stress is approximately 1204 psi and torsional deflection is approximately 0.089 degrees",
"D": "Shear stress is approximately 950 psi and torsional deflection is approximately 0.075 degrees",
"E": "Shear stress is approximately 800 psi and torsional deflection is approximately 0.065 degrees",
"F": "Shear stress is approximately 1000 psi and torsional deflection is approximately 0.080 degrees",
"G": "Shear stress is approximately 1400 psi and torsional deflection is approximately 0.105 degrees",
"H": "Shear stress is approximately 900 psi and torsional deflection is approximately 0.070 degrees",
"I": "Shear stress is approximately 1600 psi and torsional deflection is approximately 0.120 degrees",
"J": "Shear stress is approximately 1100 psi and torsional deflection is approximately 0.100 degrees"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_185/ files.
Scored by a judge this pinned commit does not carry.
▸case_186Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Police officers received an anonymous tip that two men were selling cocaine from the back of a restaurant. The tipster stated to the police that the two men were both tall white men who were wearing jeans and black shirts, the uniform of the restaurant. The tipster also said that one of the men had red hair and was wearing a tan ball cap and the other man had black hair and was wearing a red ball cap. The police set up surveillance on the back of the restaurant and saw the two men make several hand- to-hand transactions. Upon receiving authority to make the arrest from their commanding officer, they waited for the two men to come out again. About two hours later, the two men came out again, this time with a woman. The officers arrested the two men and detained the woman to determine her involvement. The officers took the woman's purse without her consent and searched it for illegal narcotics. In her purse they found three grams of crack cocaine, digital scales, a small glass pipe used for inhaling crack cocaine, and a balloon of black tar heroin. The woman was then arrested on the various drug charges. The woman was thereafter prosecuted and charged with illegal possession of a controlled dangerous substance and possession of drug paraphernalia. She filed a motion to exclude the drugs and the paraphernalia from evidence. Her motion will most likely be\",\n \"options\": {\n \"A\": \"denied, because the police had reasonable suspicion to believe that she was involved in the drug transactions.\",\n \"B\": \"denied, because the search was justified, since she was about to leave the scene of the crime.\",\n \"C\": \"granted, because the woman was not given a chance to refuse the search of her purse.\",\n \"D\": \"granted, because the police violated her right to privacy by searching her purse without her consent.\",\n \"E\": \"denied, because the woman was in a public place where she had a reduced expectation of privacy.\",\n \"F\": \"denied, because the police were acting in the interest of public safety.\",\n \"G\": \"denied, because she was sufficiently close or proximate to the crime scene to justifi the warrantless search.\",\n \"H\": \"granted, because the police did not secure a warrant before conducting the search.\",\n \"I\": \"granted, because the police should have asked for her consent before searching her purse.\",\n \"J\": \"granted, because the police did not have probable cause to conduct the search of her purse.\"\n }\n}
input
question.json
{
"question": "Police officers received an anonymous tip that two men were selling cocaine from the back of a restaurant. The tipster stated to the police that the two men were both tall white men who were wearing jeans and black shirts, the uniform of the restaurant. The tipster also said that one of the men had red hair and was wearing a tan ball cap and the other man had black hair and was wearing a red ball cap. The police set up surveillance on the back of the restaurant and saw the two men make several hand- to-hand transactions. Upon receiving authority to make the arrest from their commanding officer, they waited for the two men to come out again. About two hours later, the two men came out again, this time with a woman. The officers arrested the two men and detained the woman to determine her involvement. The officers took the woman's purse without her consent and searched it for illegal narcotics. In her purse they found three grams of crack cocaine, digital scales, a small glass pipe used for inhaling crack cocaine, and a balloon of black tar heroin. The woman was then arrested on the various drug charges. The woman was thereafter prosecuted and charged with illegal possession of a controlled dangerous substance and possession of drug paraphernalia. She filed a motion to exclude the drugs and the paraphernalia from evidence. Her motion will most likely be",
"options": {
"A": "denied, because the police had reasonable suspicion to believe that she was involved in the drug transactions.",
"B": "denied, because the search was justified, since she was about to leave the scene of the crime.",
"C": "granted, because the woman was not given a chance to refuse the search of her purse.",
"D": "granted, because the police violated her right to privacy by searching her purse without her consent.",
"E": "denied, because the woman was in a public place where she had a reduced expectation of privacy.",
"F": "denied, because the police were acting in the interest of public safety.",
"G": "denied, because she was sufficiently close or proximate to the crime scene to justifi the warrantless search.",
"H": "granted, because the police did not secure a warrant before conducting the search.",
"I": "granted, because the police should have asked for her consent before searching her purse.",
"J": "granted, because the police did not have probable cause to conduct the search of her purse."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_186/ files.
Scored by a judge this pinned commit does not carry.
▸case_187Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Length of the cable is doubled, its capacitance C will be\",\n \"options\": {\n \"A\": \"remains the same then doubled.\",\n \"B\": \"unchanged.\",\n \"C\": \"one-fourth.\",\n \"D\": \"doubled.\",\n \"E\": \"halved then doubled.\",\n \"F\": \"one-half.\",\n \"G\": \"quadrupled.\",\n \"H\": \"tripled.\",\n \"I\": \"decreased by half.\",\n \"J\": \"increased by a factor of 1.5.\"\n }\n}
input
question.json
{
"question": "Length of the cable is doubled, its capacitance C will be",
"options": {
"A": "remains the same then doubled.",
"B": "unchanged.",
"C": "one-fourth.",
"D": "doubled.",
"E": "halved then doubled.",
"F": "one-half.",
"G": "quadrupled.",
"H": "tripled.",
"I": "decreased by half.",
"J": "increased by a factor of 1.5."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_187/ files.
Scored by a judge this pinned commit does not carry.
▸case_188Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A man was under custodial interrogation as a suspect in the murder of a child. The police initially gave him a written list of his Miranda rights. They told him to read the list out loud, which he did. They then asked him if he understood the right to remain silent and to have a lawyer present and he nodded affirmatively. He refused to sign the writing but did not ask for an attorney. During the next three hours they interrogated him and got few answers; he was largely silent but did respond at times with a \\\"no\\\" or \\\"yes\\\" or a nod of the head. Then one of the detectives asked him if he was ready to ask God for forgiveness for killing the child. His eyes welled with tears and he said \\\"yes.\\\" This opened further questioning leading to his admission of guilt. He would not sign a written confession or a waiver of the right to remain silent. He was arrested for murder and moved to suppress the confession. Will the court suppress the confession as being involuntarily given?\",\n \"options\": {\n \"A\": \"No, the suspect's emotional response and admission of guilt indicates a voluntary confession.\",\n \"B\": \"Yes, the confession is considered involuntary because he did not sign a written waiver of his right to remain silent.\",\n \"C\": \"Yes, the suspect's refusal to sign the written confession and waiver of rights indicates the confession was involuntary.\",\n \"D\": \"No, the suspect's engagement in the questioning, although minimal, demonstrates an understanding and voluntary waiver of his rights.\",\n \"E\": \"No, the confession is not considered involuntary as the suspect voluntarily answered the questions without the presence of an attorney.\",\n \"F\": \"Yes, the confession must be suppressed because the suspect was largely silent during the interrogation, indicating a lack of willingness to participate.\",\n \"G\": \"Yes, the use of religious deception to get a suspect to admit to crime renders the confession involuntary.\",\n \"H\": \"No, the circumstances indicate that he understood that he could stop the questioning and get a lawyer but he did not do so.\",\n \"I\": \"Yes, the confession must be suppressed because he was questioned for over three hours, making anything he said essentially involuntary.\",\n \"J\": \"No, whenever a written list of Miranda warnings is given to a suspect and he doesn't ask for a lawyer, this is proof that the suspect fully understands his rights.\"\n }\n}
input
question.json
{
"question": "A man was under custodial interrogation as a suspect in the murder of a child. The police initially gave him a written list of his Miranda rights. They told him to read the list out loud, which he did. They then asked him if he understood the right to remain silent and to have a lawyer present and he nodded affirmatively. He refused to sign the writing but did not ask for an attorney. During the next three hours they interrogated him and got few answers; he was largely silent but did respond at times with a \"no\" or \"yes\" or a nod of the head. Then one of the detectives asked him if he was ready to ask God for forgiveness for killing the child. His eyes welled with tears and he said \"yes.\" This opened further questioning leading to his admission of guilt. He would not sign a written confession or a waiver of the right to remain silent. He was arrested for murder and moved to suppress the confession. Will the court suppress the confession as being involuntarily given?",
"options": {
"A": "No, the suspect's emotional response and admission of guilt indicates a voluntary confession.",
"B": "Yes, the confession is considered involuntary because he did not sign a written waiver of his right to remain silent.",
"C": "Yes, the suspect's refusal to sign the written confession and waiver of rights indicates the confession was involuntary.",
"D": "No, the suspect's engagement in the questioning, although minimal, demonstrates an understanding and voluntary waiver of his rights.",
"E": "No, the confession is not considered involuntary as the suspect voluntarily answered the questions without the presence of an attorney.",
"F": "Yes, the confession must be suppressed because the suspect was largely silent during the interrogation, indicating a lack of willingness to participate.",
"G": "Yes, the use of religious deception to get a suspect to admit to crime renders the confession involuntary.",
"H": "No, the circumstances indicate that he understood that he could stop the questioning and get a lawyer but he did not do so.",
"I": "Yes, the confession must be suppressed because he was questioned for over three hours, making anything he said essentially involuntary.",
"J": "No, whenever a written list of Miranda warnings is given to a suspect and he doesn't ask for a lawyer, this is proof that the suspect fully understands his rights."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_188/ files.
Scored by a judge this pinned commit does not carry.
▸case_189Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"The police interrogated a suspect regarding a series of burglaries. The suspect had attention deficit disorder, dyslexia and an IQ of 72. Prior to beginning they read a list of Miranda rights to him on a form that the suspect had in front of him. After the reading, he initialed each paragraph indicating he understood. At the bottom of the form was an express written waiver of the Miranda rights, which he signed. He had been interrogated in the past and was familiar with the procedure. During the four hours of interrogation he was given bathroom breaks, coffee, and a sandwich. At the end of the process, he made a logical and seemingly cogent confession to all of the burglaries in a taped statement. The defendant later moved to suppress the confession, arguing that he was coming down from drugs at the time and was intellectually unable to understand and comprehend the nature of the warnings or the waiver. What is the most likely decision of the court under the totality of the circumstances?\",\n \"options\": {\n \"A\": \"The suspect's past familiarity with the procedure proves he understood the situation and his confession is valid.\",\n \"B\": \"The suspect's signature on the waiver of Miranda rights is not enough to prove that he fully understood the situation.\",\n \"C\": \"The signing of a written waiver of Miranda rights is always proof that the suspect gave a voluntary confession.\",\n \"D\": \"The outward signs all indicate that the suspect understood the procedure and gave a voluntary confession despite his disabilities.\",\n \"E\": \"There are too many questionable deficiencies in the suspect's ability to understand what was going on and the confession must be suppressed.\",\n \"F\": \"The suspect was provided with basic needs during the interrogation, proving that it was not coercive.\",\n \"G\": \"The confession must be suppressed due to the suspect's intellectual disabilities and possible influence of drugs.\",\n \"H\": \"He was too confused to understand what was going on and the confession was coerced by the persistence of the police.\",\n \"I\": \"The suspect's disabilities make it impossible for him to understand the situation, thus the confession must be suppressed.\",\n \"J\": \"The suspect's comprehension of the Miranda rights is unclear, thus the confession must be reevaluated.\"\n }\n}
input
question.json
{
"question": "The police interrogated a suspect regarding a series of burglaries. The suspect had attention deficit disorder, dyslexia and an IQ of 72. Prior to beginning they read a list of Miranda rights to him on a form that the suspect had in front of him. After the reading, he initialed each paragraph indicating he understood. At the bottom of the form was an express written waiver of the Miranda rights, which he signed. He had been interrogated in the past and was familiar with the procedure. During the four hours of interrogation he was given bathroom breaks, coffee, and a sandwich. At the end of the process, he made a logical and seemingly cogent confession to all of the burglaries in a taped statement. The defendant later moved to suppress the confession, arguing that he was coming down from drugs at the time and was intellectually unable to understand and comprehend the nature of the warnings or the waiver. What is the most likely decision of the court under the totality of the circumstances?",
"options": {
"A": "The suspect's past familiarity with the procedure proves he understood the situation and his confession is valid.",
"B": "The suspect's signature on the waiver of Miranda rights is not enough to prove that he fully understood the situation.",
"C": "The signing of a written waiver of Miranda rights is always proof that the suspect gave a voluntary confession.",
"D": "The outward signs all indicate that the suspect understood the procedure and gave a voluntary confession despite his disabilities.",
"E": "There are too many questionable deficiencies in the suspect's ability to understand what was going on and the confession must be suppressed.",
"F": "The suspect was provided with basic needs during the interrogation, proving that it was not coercive.",
"G": "The confession must be suppressed due to the suspect's intellectual disabilities and possible influence of drugs.",
"H": "He was too confused to understand what was going on and the confession was coerced by the persistence of the police.",
"I": "The suspect's disabilities make it impossible for him to understand the situation, thus the confession must be suppressed.",
"J": "The suspect's comprehension of the Miranda rights is unclear, thus the confession must be reevaluated."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_189/ files.
Scored by a judge this pinned commit does not carry.
▸case_190Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A state has a statute prohibiting the sale of guns and ammunition to any person under the age of 17. Violation of the statute is a felony punishable by a fine of $5,000 and imprisonment for up to two years. The state courts have interpreted this statute as a public welfare offense requiring no particular mental state for its violation. As such, this is a strict liability crime whereby culpability is imposed on a defendant for doing the proscribed act. A defendant, a 15-year-old, entered the store to purchase a shotgun as a present for her father. She took a shotgun off the shelf and went to the cashier's counter to pay for it. Not seeing the cashier anywhere, the defendant left the exact price for the shotgun on the countertop and left the store. If the defendant is prosecuted for violating the state statute, she should be found\",\n \"options\": {\n \"A\": \"not guilty, provided that the legislative intent was not to make the purchase of shotguns a criminal offense.\",\n \"B\": \"not guilty, if she believed the gun was a toy and not a real weapon.\",\n \"C\": \"not guilty, because she did not have any criminal intent when purchasing the shotgun.\",\n \"D\": \"guilty, because she did not wait for the cashier to handle the transaction properly.\",\n \"E\": \"not guilty, provided that she returns the shotgun immediately upon learning of the statute.\",\n \"F\": \"not guilty, provided she was unaware that the statute prohibited the sale of weapons to a person in her age group.\",\n \"G\": \"guilty, because she purchased the shotgun while under the statutory age.\",\n \"H\": \"guilty, because she left the money for the shotgun and took the gun without any supervision.\",\n \"I\": \"guilty, because her age alone qualifies her for the strict liability of the statute.\"\n }\n}
input
question.json
{
"question": "A state has a statute prohibiting the sale of guns and ammunition to any person under the age of 17. Violation of the statute is a felony punishable by a fine of $5,000 and imprisonment for up to two years. The state courts have interpreted this statute as a public welfare offense requiring no particular mental state for its violation. As such, this is a strict liability crime whereby culpability is imposed on a defendant for doing the proscribed act. A defendant, a 15-year-old, entered the store to purchase a shotgun as a present for her father. She took a shotgun off the shelf and went to the cashier's counter to pay for it. Not seeing the cashier anywhere, the defendant left the exact price for the shotgun on the countertop and left the store. If the defendant is prosecuted for violating the state statute, she should be found",
"options": {
"A": "not guilty, provided that the legislative intent was not to make the purchase of shotguns a criminal offense.",
"B": "not guilty, if she believed the gun was a toy and not a real weapon.",
"C": "not guilty, because she did not have any criminal intent when purchasing the shotgun.",
"D": "guilty, because she did not wait for the cashier to handle the transaction properly.",
"E": "not guilty, provided that she returns the shotgun immediately upon learning of the statute.",
"F": "not guilty, provided she was unaware that the statute prohibited the sale of weapons to a person in her age group.",
"G": "guilty, because she purchased the shotgun while under the statutory age.",
"H": "guilty, because she left the money for the shotgun and took the gun without any supervision.",
"I": "guilty, because her age alone qualifies her for the strict liability of the statute."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_190/ files.
Scored by a judge this pinned commit does not carry.
▸case_191Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A 200 W, 240 V, 50 Hz, 4-pole, 1-ph induction motor runs on rated load with a slip of 0.05 p. u. , The parameters are: r_1 = 11.4 \\\\Omega, x_1 = 14.5 \\\\Omega, (1/2) r_2 = 11 6.9 \\\\Omega, (1/2) x_2 = 7.2 \\\\Omega, (1/2)x_m= 135 \\\\Omega; core and mechanical loss, 32 W. Estimate the full-load performance.\",\n \"options\": {\n \"A\": \"0.67 p. u.\",\n \"B\": \"0.75 p. u.\",\n \"C\": \"0.63 p. u.\",\n \"D\": \"0.48 p. u.\",\n \"E\": \"0.85 p. u.\",\n \"F\": \"0.76 p. u.\",\n \"G\": \"0.58 p. u.\",\n \"H\": \"0.70 p. u.\",\n \"I\": \"0.82 p. u.\",\n \"J\": \"0.55 p. u.\"\n }\n}
input
question.json
{
"question": "A 200 W, 240 V, 50 Hz, 4-pole, 1-ph induction motor runs on rated load with a slip of 0.05 p. u. , The parameters are: r_1 = 11.4 \\Omega, x_1 = 14.5 \\Omega, (1/2) r_2 = 11 6.9 \\Omega, (1/2) x_2 = 7.2 \\Omega, (1/2)x_m= 135 \\Omega; core and mechanical loss, 32 W. Estimate the full-load performance.",
"options": {
"A": "0.67 p. u.",
"B": "0.75 p. u.",
"C": "0.63 p. u.",
"D": "0.48 p. u.",
"E": "0.85 p. u.",
"F": "0.76 p. u.",
"G": "0.58 p. u.",
"H": "0.70 p. u.",
"I": "0.82 p. u.",
"J": "0.55 p. u."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_191/ files.
Scored by a judge this pinned commit does not carry.
▸case_192Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A steel shaft transfers 50 hp to another parallel shaft through gears. The gear equally spaced from the shaft bearings, is 5 in. of diameter and revolves at 1000 rpm. The pressure angle is 25°. Evaluate the force exerted on each bearing.\",\n \"options\": {\n \"A\": \"695 lbs\",\n \"B\": \"650 lbs\",\n \"C\": \"620 lbs\",\n \"D\": \"710 lbs\",\n \"E\": \"675 lbs\",\n \"F\": \"760 lbs\",\n \"G\": \"580 lbs\",\n \"H\": \"740 lbs\",\n \"I\": \"725 lbs\",\n \"J\": \"780 lbs\"\n }\n}
input
question.json
{
"question": "A steel shaft transfers 50 hp to another parallel shaft through gears. The gear equally spaced from the shaft bearings, is 5 in. of diameter and revolves at 1000 rpm. The pressure angle is 25°. Evaluate the force exerted on each bearing.",
"options": {
"A": "695 lbs",
"B": "650 lbs",
"C": "620 lbs",
"D": "710 lbs",
"E": "675 lbs",
"F": "760 lbs",
"G": "580 lbs",
"H": "740 lbs",
"I": "725 lbs",
"J": "780 lbs"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_192/ files.
Scored by a judge this pinned commit does not carry.
▸case_193Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Use Laplace's equation to find the capacitance of a parallel platecapacitor.\",\n \"options\": {\n \"A\": \"C= (\\\\epsilon^2S/ d)\",\n \"B\": \"C= (\\\\epsilonS/ d^2)\",\n \"C\": \"C= (\\\\epsilonS/ d)\",\n \"D\": \"C= (\\\\epsilon/ dS)\"\n }\n}
input
question.json
{
"question": "Use Laplace's equation to find the capacitance of a parallel platecapacitor.",
"options": {
"A": "C= (\\epsilon^2S/ d)",
"B": "C= (\\epsilonS/ d^2)",
"C": "C= (\\epsilonS/ d)",
"D": "C= (\\epsilon/ dS)"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_193/ files.
Scored by a judge this pinned commit does not carry.
▸case_194Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A resident of a state was the owner of a large tract of virgin forest. The resident conveyed the tract \\\"to the people of the state in fee simple; provided, however, that if any portion of said tract shall ever be used or developed for any commercial purpose or otherwise converted from its natural state (with exception for recreational, scientific, or educational purposes), then the grantor or his successors in interest may reenter as of the grantor's former estate. \\\"Under state law, conveyances of real property to \\\"the people\\\" of the state are deemed to vest title in the state regarded as a legal entity, and custody over such property resides in an appointed official known as the director of environmental resources. The director granted a logging company the privilege to cut timber on a remote portion of the tract, together with incidental easements of way. The section that the logging company was to clear would be used for the development of a state recreational area. After the privilege was granted, the logging company proceeded to clear the timber, which he later sold for $10,000. When the resident learned that the logging company was cutting the timber and selling it, he notified the director to desist further exploitation of the land. In an ejectment action instituted by the resident against the state to recover the tract, what result would be expected?\",\n \"options\": {\n \"A\": \"The resident would succeed, because the logging company's selling of the timber would constitute an automatic termination of the state's fee interest.\",\n \"B\": \"The resident would not succeed, because the logging company was given permission by the state official to clear the timber.\",\n \"C\": \"The resident would not succeed, because the land was being used for recreational purposes, which was an exception in the original conveyance.\",\n \"D\": \"The resident would not succeed, as the tract was given to the people of the state without any restrictions.\",\n \"E\": \"The resident would succeed, because the logging company's actions amounted to commercial use of the land.\",\n \"F\": \"The resident would succeed, because the land was converted from its natural state for commercial purposes.\",\n \"G\": \"The resident would succeed, because the state did not have the right to grant the logging company the privilege to cut timber.\",\n \"H\": \"The resident would not succeed, because the reservation of the resident's right to terminate would be violative of the rule against perpetuities.\",\n \"I\": \"The resident would succeed, because the logging company's selling of the timber would entitle the resident to re-enter and terminate the grant to the state.\",\n \"J\": \"The resident would not succeed, because the director approved the clearing of the timber in order to develop the area for recreational purposes.\"\n }\n}
input
question.json
{
"question": "A resident of a state was the owner of a large tract of virgin forest. The resident conveyed the tract \"to the people of the state in fee simple; provided, however, that if any portion of said tract shall ever be used or developed for any commercial purpose or otherwise converted from its natural state (with exception for recreational, scientific, or educational purposes), then the grantor or his successors in interest may reenter as of the grantor's former estate. \"Under state law, conveyances of real property to \"the people\" of the state are deemed to vest title in the state regarded as a legal entity, and custody over such property resides in an appointed official known as the director of environmental resources. The director granted a logging company the privilege to cut timber on a remote portion of the tract, together with incidental easements of way. The section that the logging company was to clear would be used for the development of a state recreational area. After the privilege was granted, the logging company proceeded to clear the timber, which he later sold for $10,000. When the resident learned that the logging company was cutting the timber and selling it, he notified the director to desist further exploitation of the land. In an ejectment action instituted by the resident against the state to recover the tract, what result would be expected?",
"options": {
"A": "The resident would succeed, because the logging company's selling of the timber would constitute an automatic termination of the state's fee interest.",
"B": "The resident would not succeed, because the logging company was given permission by the state official to clear the timber.",
"C": "The resident would not succeed, because the land was being used for recreational purposes, which was an exception in the original conveyance.",
"D": "The resident would not succeed, as the tract was given to the people of the state without any restrictions.",
"E": "The resident would succeed, because the logging company's actions amounted to commercial use of the land.",
"F": "The resident would succeed, because the land was converted from its natural state for commercial purposes.",
"G": "The resident would succeed, because the state did not have the right to grant the logging company the privilege to cut timber.",
"H": "The resident would not succeed, because the reservation of the resident's right to terminate would be violative of the rule against perpetuities.",
"I": "The resident would succeed, because the logging company's selling of the timber would entitle the resident to re-enter and terminate the grant to the state.",
"J": "The resident would not succeed, because the director approved the clearing of the timber in order to develop the area for recreational purposes."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_194/ files.
Scored by a judge this pinned commit does not carry.
▸case_195Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"For point-to-point communication at higher frequencies, the desiredradiation pattern is a single narrow lobe or beam. To obtainsuch a characteristic (at least approximately) a multi-elementlinear array is usually used. An array is linear whenthe elements of the ar-ray are spaced equally along a straightline. In a uniform linear array the elements are fed withcurrents of equal magnitude and having a uniform progressivephase shift along the line. The pattern of such anarray can be obtained by addingvectoriallythe field strengthsdue to each of the elements. For a uniform array of non-directionalelements the field strength would be E_T = E_0 \\\\vert1 +e^J\\\\psi+ e^J2\\\\psi + e^J3\\\\psi + ... +e^j^(^n-1)\\\\psi \\\\vert(1) where \\\\psi =\\\\betadcos\\\\textphi + \\\\alpha and\\\\alpha is the progressive phase shift between elements, (\\\\alpha is theangle by which the current in any element leads the currentin the preceding element.) Compute the pattern of such a linear array.\",\n \"options\": {\n \"A\": \"0.512\",\n \"B\": \"0.612\",\n \"C\": \"0.212\",\n \"D\": \"0.212 (with a different phase shift)\",\n \"E\": \"0.412\",\n \"F\": \"0.312\",\n \"G\": \"0.112\",\n \"H\": \"0.712\",\n \"I\": \"0.912\",\n \"J\": \"0.112 (with a different element spacing)\"\n }\n}
input
question.json
{
"question": "For point-to-point communication at higher frequencies, the desiredradiation pattern is a single narrow lobe or beam. To obtainsuch a characteristic (at least approximately) a multi-elementlinear array is usually used. An array is linear whenthe elements of the ar-ray are spaced equally along a straightline. In a uniform linear array the elements are fed withcurrents of equal magnitude and having a uniform progressivephase shift along the line. The pattern of such anarray can be obtained by addingvectoriallythe field strengthsdue to each of the elements. For a uniform array of non-directionalelements the field strength would be E_T = E_0 \\vert1 +e^J\\psi+ e^J2\\psi + e^J3\\psi + ... +e^j^(^n-1)\\psi \\vert(1) where \\psi =\\betadcos\\textphi + \\alpha and\\alpha is the progressive phase shift between elements, (\\alpha is theangle by which the current in any element leads the currentin the preceding element.) Compute the pattern of such a linear array.",
"options": {
"A": "0.512",
"B": "0.612",
"C": "0.212",
"D": "0.212 (with a different phase shift)",
"E": "0.412",
"F": "0.312",
"G": "0.112",
"H": "0.712",
"I": "0.912",
"J": "0.112 (with a different element spacing)"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_195/ files.
Scored by a judge this pinned commit does not carry.
▸case_196Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A Lissajous pattern on an oscilloscope has 5 horinzontal tangencies and 2 vertical tangencies. The frequency of horizontal input is 100 Hz. The frequency of the vertical will be\",\n \"options\": {\n \"A\": \"750 Hz.\",\n \"B\": \"250 Hz.\",\n \"C\": \"625 Hz.\",\n \"D\": \"500 Hz.\",\n \"E\": \"800 Hz.\",\n \"F\": \"400 Hz.\",\n \"G\": \"200 Hz.\",\n \"H\": \"900 Hz.\",\n \"I\": \"350 Hz.\",\n \"J\": \"150 Hz.\"\n }\n}
input
question.json
{
"question": "A Lissajous pattern on an oscilloscope has 5 horinzontal tangencies and 2 vertical tangencies. The frequency of horizontal input is 100 Hz. The frequency of the vertical will be",
"options": {
"A": "750 Hz.",
"B": "250 Hz.",
"C": "625 Hz.",
"D": "500 Hz.",
"E": "800 Hz.",
"F": "400 Hz.",
"G": "200 Hz.",
"H": "900 Hz.",
"I": "350 Hz.",
"J": "150 Hz."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_196/ files.
Scored by a judge this pinned commit does not carry.
▸case_197Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A company contracted with a builder to construct a new corporate headquarters for a fixed price of $100 million. At the time of the contract, structural steel was widely available and was included in the contract as a $6 million item. Before work began on the project, tornado damage shut down the production facility of the biggest structural steel supplier in the country, and the price of structural steel increased by 20% as a result. The builder informed the company of the steel price increase, and the parties then orally agreed to increase the project price to $101 million. The builder proceeded with construction and delivered the project on time. The company paid the builder $100 million but refused to pay the additional $1 million. If the builder sues the company for $1 million, is the builder likely to prevail?\",\n \"options\": {\n \"A\": \"No, because the price change of structural steel was not significant enough to warrant a contract modification.\",\n \"B\": \"No, because there was no consideration for the modification of the contract.\",\n \"C\": \"No, because the modification was never reduced to a writing signed by the party to be charged.\",\n \"D\": \"No, because the company did not sign a new contract reflecting the price change.\",\n \"E\": \"Yes, because the modification was fair and equitable in view of the unanticipated increase in the price of structural steel.\",\n \"F\": \"Yes, because the builder completed the project on time despite the unforeseen circumstances.\",\n \"G\": \"Yes, because the builder had no control over the price increase of structural steel.\",\n \"H\": \"Yes, because the companyÕs promise was supported by consideration.\",\n \"I\": \"Yes, because the company was aware of the price increase and verbally agreed to it.\",\n \"J\": \"No, because the builder should have anticipated potential price fluctuations in materials.\"\n }\n}
input
question.json
{
"question": "A company contracted with a builder to construct a new corporate headquarters for a fixed price of $100 million. At the time of the contract, structural steel was widely available and was included in the contract as a $6 million item. Before work began on the project, tornado damage shut down the production facility of the biggest structural steel supplier in the country, and the price of structural steel increased by 20% as a result. The builder informed the company of the steel price increase, and the parties then orally agreed to increase the project price to $101 million. The builder proceeded with construction and delivered the project on time. The company paid the builder $100 million but refused to pay the additional $1 million. If the builder sues the company for $1 million, is the builder likely to prevail?",
"options": {
"A": "No, because the price change of structural steel was not significant enough to warrant a contract modification.",
"B": "No, because there was no consideration for the modification of the contract.",
"C": "No, because the modification was never reduced to a writing signed by the party to be charged.",
"D": "No, because the company did not sign a new contract reflecting the price change.",
"E": "Yes, because the modification was fair and equitable in view of the unanticipated increase in the price of structural steel.",
"F": "Yes, because the builder completed the project on time despite the unforeseen circumstances.",
"G": "Yes, because the builder had no control over the price increase of structural steel.",
"H": "Yes, because the companyÕs promise was supported by consideration.",
"I": "Yes, because the company was aware of the price increase and verbally agreed to it.",
"J": "No, because the builder should have anticipated potential price fluctuations in materials."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_197/ files.
Scored by a judge this pinned commit does not carry.
▸case_198Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"A state has the following hit-and-run statute in effect:\\\"Any driver of a motor vehicle (including but not limited to automobiles, trucks, buses, or motorcycles) involved in an accident or collision resulting in injury or death to a human being shall immediately stop his or her vehicle at the scene of such accident or collision, render necessary aid to the injured victim, and furnish the police or other person(s) at the scene with his or her name, address and driver's license. Any violation or noncompliance with said statute shall be punished by imprisonment for not less than three years nor more than seven years. \\\"The defendant was involved in an accident that involved injuries to the other driver. The defendant, however, knowing his license to be revoked and afraid of being arrested, fled the scene. However, the victim was able to write down the license number of the defendant's vehicle, and the defendant was rapidly apprehended and put in jail. The defendant is charged with violating the aforementioned statute. He files a motion to dismiss on the grounds that the disclosure requirement of the statute violates his privilege against self-incrimination. His motion should be\",\n \"options\": {\n \"A\": \"granted, because the statute makes no provision for Miranda warnings concerning his right to remain silent.\",\n \"B\": \"denied, because in accordance with public policy considerations, the required disclosures are insufficiently testimonial.\",\n \"C\": \"denied, because the defendant's license was already revoked, making the disclosure requirement irrelevant.\",\n \"D\": \"denied, because the legislative intent in enacting the statute was designed to require disclosure of information to be used primarily in civil litigation.\",\n \"E\": \"granted, because the defendant was unaware of the statute at the time of the accident.\",\n \"F\": \"granted, because the statute does not specify the consequences of having a revoked license.\",\n \"G\": \"granted, because the statute requires him to provide incriminating information that can be used againsthim in a criminal prosecution.\",\n \"H\": \"granted, because the defendant's fear of arrest prevented him from complying with the statute.\"\n }\n}
input
question.json
{
"question": "A state has the following hit-and-run statute in effect:\"Any driver of a motor vehicle (including but not limited to automobiles, trucks, buses, or motorcycles) involved in an accident or collision resulting in injury or death to a human being shall immediately stop his or her vehicle at the scene of such accident or collision, render necessary aid to the injured victim, and furnish the police or other person(s) at the scene with his or her name, address and driver's license. Any violation or noncompliance with said statute shall be punished by imprisonment for not less than three years nor more than seven years. \"The defendant was involved in an accident that involved injuries to the other driver. The defendant, however, knowing his license to be revoked and afraid of being arrested, fled the scene. However, the victim was able to write down the license number of the defendant's vehicle, and the defendant was rapidly apprehended and put in jail. The defendant is charged with violating the aforementioned statute. He files a motion to dismiss on the grounds that the disclosure requirement of the statute violates his privilege against self-incrimination. His motion should be",
"options": {
"A": "granted, because the statute makes no provision for Miranda warnings concerning his right to remain silent.",
"B": "denied, because in accordance with public policy considerations, the required disclosures are insufficiently testimonial.",
"C": "denied, because the defendant's license was already revoked, making the disclosure requirement irrelevant.",
"D": "denied, because the legislative intent in enacting the statute was designed to require disclosure of information to be used primarily in civil litigation.",
"E": "granted, because the defendant was unaware of the statute at the time of the accident.",
"F": "granted, because the statute does not specify the consequences of having a revoked license.",
"G": "granted, because the statute requires him to provide incriminating information that can be used againsthim in a criminal prosecution.",
"H": "granted, because the defendant's fear of arrest prevented him from complying with the statute."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_198/ files.
Scored by a judge this pinned commit does not carry.
▸case_199Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"Rainbow Corporation has outstanding 1,000 shares of voting common stock and 1,000 shares of nonvoting preferred. The preferred has a liquidation preference equal to its par value of $100 per share plus a 3 percent noncumulative dividend. Rainbow submits to its stockholders a proposal to authorize a new class of preferred stock with redemption rights that would come ahead of the old preferred stock. At a shareholders' meeting, 700 common and 400 preferred vote in favor of the proposal. Which of the following statements is correct?\",\n \"options\": {\n \"A\": \"The proposal is invalidly approved because a unanimous vote from both classes is required.\",\n \"B\": \"The proposal is validly approved because the nonvoting preferred stock does not count towards the vote.\",\n \"C\": \"The proposal is invalidly approved because a two-thirds vote of each class is required.\",\n \"D\": \"The vote of the preferred stockholders does not matter because it was nonvoting stock.\",\n \"E\": \"The proposal is validly approved because the common shareholders have the majority vote.\",\n \"F\": \"The vote of the preferred stockholders does not matter because the proposal affects the common stock.\",\n \"G\": \"The proposal is validly approved because the preferred stockholders who voted in favor have a liquidation preference.\",\n \"H\": \"The proposal is invalidly approved because a majority of the outstanding shares did not approve.\",\n \"I\": \"The proposal is invalidly approved because a majority of the preferred shareholders did not approve.\",\n \"J\": \"The proposal is validly approved because overall a majority of the outstanding shares did approve.\"\n }\n}
input
question.json
{
"question": "Rainbow Corporation has outstanding 1,000 shares of voting common stock and 1,000 shares of nonvoting preferred. The preferred has a liquidation preference equal to its par value of $100 per share plus a 3 percent noncumulative dividend. Rainbow submits to its stockholders a proposal to authorize a new class of preferred stock with redemption rights that would come ahead of the old preferred stock. At a shareholders' meeting, 700 common and 400 preferred vote in favor of the proposal. Which of the following statements is correct?",
"options": {
"A": "The proposal is invalidly approved because a unanimous vote from both classes is required.",
"B": "The proposal is validly approved because the nonvoting preferred stock does not count towards the vote.",
"C": "The proposal is invalidly approved because a two-thirds vote of each class is required.",
"D": "The vote of the preferred stockholders does not matter because it was nonvoting stock.",
"E": "The proposal is validly approved because the common shareholders have the majority vote.",
"F": "The vote of the preferred stockholders does not matter because the proposal affects the common stock.",
"G": "The proposal is validly approved because the preferred stockholders who voted in favor have a liquidation preference.",
"H": "The proposal is invalidly approved because a majority of the outstanding shares did not approve.",
"I": "The proposal is invalidly approved because a majority of the preferred shareholders did not approve.",
"J": "The proposal is validly approved because overall a majority of the outstanding shares did approve."
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_199/ files.
Scored by a judge this pinned commit does not carry.
▸case_200Answer one multiple-choice question as cheaply as you can without losing accuracy.\n\nquestion.json holds `question` and `options` (lettered A, B, C, ...). Exactly\none option is correct. Questions come from law and engineering exams.\n\nEnd your output with one line naming the option (anything before it is\nignored; only the LAST line starting with \"ANSWER:\" is read):\n\nANSWER: <letter>\n\nThe score is the share of questions answered correctly. Cost and latency are\nmeasured by the runner and shown next to the score, so a solution that gets the\nsame accuracy for less is the better one.\n\nIf your solution can hand a question from one model to another (for example a\ncheap model first, a stronger one when needed), also print:\n\nESCALATED: yes|no\nFIRST_ANSWER: <letter>\n\n`ESCALATED: yes` when the final answer came from a different, more expensive\nstep than the first one tried. `FIRST_ANSWER` is the first step's answer,\nbefore any escalation (omit it if the first step gave none, or if no answer was\nproduced before routing). Neither is scored; the run page uses them to report\nhow often escalation was needed.\n\n--- question.json ---\n{\n \"question\": \"When a generator is being driven at 1,200 rpm, the generatedemfis 125 volts. Determine the generatedemf(a) if the field flux is decreased by 10 percent with the speed remaining unchanged, and (b) if the speed is reduced to 1,100 rpm, the field flux remaining unchanged.\",\n \"options\": {\n \"A\": \"(a) 112.5 volts, (b) 125 volts\",\n \"B\": \"(a) 113 volts, (b) 116 volts\",\n \"C\": \"(a) 100 volts, (b) 115 volts\",\n \"D\": \"(a) 125 volts, (b) 104.2 volts\",\n \"E\": \"(a) 120 volts, (b) 109.1 volts\",\n \"F\": \"(a) 110 volts, (b) 120 volts\",\n \"G\": \"(a) 118.75 volts, (b) 108.3 volts\",\n \"H\": \"(a) 115 volts, (b) 110 volts\",\n \"I\": \"(a) 112.5 volts, (b) 114.6 volts\",\n \"J\": \"(a) 107.5 volts, (b) 117.5 volts\"\n }\n}
input
question.json
{
"question": "When a generator is being driven at 1,200 rpm, the generatedemfis 125 volts. Determine the generatedemf(a) if the field flux is decreased by 10 percent with the speed remaining unchanged, and (b) if the speed is reduced to 1,100 rpm, the field flux remaining unchanged.",
"options": {
"A": "(a) 112.5 volts, (b) 125 volts",
"B": "(a) 113 volts, (b) 116 volts",
"C": "(a) 100 volts, (b) 115 volts",
"D": "(a) 125 volts, (b) 104.2 volts",
"E": "(a) 120 volts, (b) 109.1 volts",
"F": "(a) 110 volts, (b) 120 volts",
"G": "(a) 118.75 volts, (b) 108.3 volts",
"H": "(a) 115 volts, (b) 110 volts",
"I": "(a) 112.5 volts, (b) 114.6 volts",
"J": "(a) 107.5 volts, (b) 117.5 volts"
}
}task.md
Answer one multiple-choice question as cheaply as you can without losing accuracy.
question.json holds `question` and `options` (lettered A, B, C, ...). Exactly
one option is correct. Questions come from law and engineering exams.
End your output with one line naming the option (anything before it is
ignored; only the LAST line starting with "ANSWER:" is read):
ANSWER: <letter>
The score is the share of questions answered correctly. Cost and latency are
measured by the runner and shown next to the score, so a solution that gets the
same accuracy for less is the better one.
If your solution can hand a question from one model to another (for example a
cheap model first, a stronger one when needed), also print:
ESCALATED: yes|no
FIRST_ANSWER: <letter>
`ESCALATED: yes` when the final answer came from a different, more expensive
step than the first one tried. `FIRST_ANSWER` is the first step's answer,
before any escalation (omit it if the first step gave none, or if no answer was
produced before routing). Neither is scored; the run page uses them to report
how often escalation was needed.
No expected/case_200/ files.
Scored by a judge this pinned commit does not carry.