reasoning benchmark
AI2 Reasoning Challenge (ARC)
A dataset of 7,787 genuine grade-school level, multiple-choice science questions assembled to encourage research in advanced question-answering. The dataset is partitioned into a Challenge Set and Easy Set, where the Challenge Set contains only questions answered incorrectly by both retrieval-based and word co-occurrence algorithms. Covers multiple scientific domains including biology, physics, earth science, and chemistry, requiring scientific reasoning, causal understanding, and conceptual knowledge beyond simple fact retrieval. Includes a supporting corpus of over 14 million science sentences.
Updated Aug 11, 2026
Models1
Model coverage1
MetricScore
EvidenceB
AI2 Reasoning Challenge (ARC) Ranking
Higher score ranks better on this benchmark.
| 1 | OP | 96.3% | 100.0% | 1 | C |
AI2 Reasoning Challenge (ARC) Highlights
The leading models and scores on this benchmark.
What is AI2 Reasoning Challenge (ARC)?
What AI2 Reasoning Challenge (ARC) measures and how its scores work.
A dataset of 7,787 genuine grade-school level, multiple-choice science questions assembled to encourage research in advanced question-answering. The dataset is partitioned into a Challenge Set and Easy Set, where the Challenge Set contains only questions answered incorrectly by both retrieval-based and word co-occurrence algorithms. Covers multiple scientific domains including biology, physics, earth science, and chemistry, requiring scientific reasoning, causal understanding, and conceptual knowledge beyond simple fact retrieval. Includes a supporting corpus of over 14 million science sentences.
Scores are shown in ratio. This benchmark is not independently verified and has an evidence level of B.
- Family
- AI2 Reasoning Challenge (ARC)
- Modality
- text
- Primary category
- reasoning
- Score direction
- higher
- LLMBoard eligible
- No
- Evaluation key
- ai2-reasoning-challenge-(arc)|llm-stats-current
Benchmark scores retain their original unit. Overall score eligibility is shown separately.
FAQ
Common questions about AI2 Reasoning Challenge (ARC).
Which model scores highest on AI2 Reasoning Challenge (ARC)?
GPT-4 is currently ranked first with 96.3%.
What does AI2 Reasoning Challenge (ARC) measure?
A dataset of 7,787 genuine grade-school level, multiple-choice science questions assembled to encourage research in advanced question-answering. The dataset is partitioned into a Challenge Set and Easy Set, where the Challenge Set contains only questions answered incorrectly by both retrieval-based and word co-occurrence algorithms. Covers multiple scientific domains including biology, physics, earth science, and chemistry, requiring scientific reasoning, causal understanding, and conceptual knowledge beyond simple fact retrieval. Includes a supporting corpus of over 14 million science sentences.
Is a higher score better?
Yes. Higher values rank better for this benchmark.
How many models are compared?
1 model results are currently shown.
Does this benchmark affect the overall score?
No. This benchmark is shown for reference but does not contribute to the overall score.