Benchmarks
210 benchmarks from 43 publishers, each with its own leaderboard. Search by name, publisher or category; every row opens the full board for that benchmark.
210 benchmarks
Metadata
| Benchmark | What it measures | Leader | Models | Weight | Source |
|---|---|---|---|---|---|
| AA Openness Index | How open a model actually is: weights, licence, training details and reproducibility. A measure of openness, not capability, so it is never part of the quality index. | K2 Think V288.9 | 34 | – | Artificial Analysis |
Reasoning
| Benchmark | What it measures | Leader | Models | Weight | Source |
|---|---|---|---|---|---|
| GPQA Diamond | Graduate-level science questions written to be Google-proof. Run by Epoch AI. | GPT-6 Astra95.8% | 285 | 1.0 | Epoch AI Benchmarking Hub |
| Humanity's Last Exam | 2,500 expert-written questions across 100+ subjects (Scale AI / CAIS). | GPT-6 Astra54.8% | 46 | 1.0 | Scale AI / CAIS |
| SimpleBench | Trick questions where humans score ~84%: spatio-temporal reasoning and social intelligence. | Claude Fable 581.9% | 88 | 1.0 | SimpleBench |
| LMArena Hard Prompts | Text arena rating on prompts judged hard. | Gemini 4 Argon1551 | 378 | 0.5 | LMArena |
| LiveBench Reasoning | LiveBench reasoning category. | GPT-6 Astra92.7% | 63 | – | LiveBench |
| GPQA Diamond (AA) | Artificial Analysis' own GPQA Diamond run. Shown for reference; Epoch's run is in the index. | GPT-6 Astra96.3% | 544 | – | Artificial Analysis |
| Humanity's Last Exam (AA) | Artificial Analysis' own HLE run. Shown for reference. | Claude Opus 5.561.4% | 584 | – | Artificial Analysis |
| GPQA Diamond (Vals) | Vals AI's own GPQA Diamond run. Shown for reference. | Gemini 3.1 Pro95.5% | 130 | – | Vals AI |
| EnigmaEval | Puzzle-hunt style multi-step reasoning. Scale AI. | Claude Fable 539.3% | 5 | 0.5 | Scale AI SEAL |
| Kagi LLM Benchmark | Kagi's unpublished reasoning, coding and instruction questions. | Claude Fable 591.4% | 136 | 0.5 | Kagi LLM Benchmark |
| GPQA Diamond (HELM) | Stanford CRFM's own GPQA Diamond run. Shown for reference; Epoch's run is in the index. | Gemini 3 Pro80.3% | 68 | – | HELM Capabilities |
| ARC-AGI-1 | Semi-private ARC-AGI-1 evaluation set. | Claude Opus 5.598.5% | 218 | 0.5 | ARC Prize |
| ARC-AGI-2 | Semi-private ARC-AGI-2 evaluation set. | GPT-6 Astra95.0% | 220 | 1.0 | ARC Prize |
| ARC-AGI-3 | Semi-private ARC-AGI-3 interactive evaluation set. | GPT-6 Astra99.9% | 53 | 0.5 | ARC Prize |
| CritPt | Unpublished physics research problems at the level of a first PhD project. Run by Artificial Analysis. | GPT-5.6 Sol32.3% | 529 | 1.0 | Artificial Analysis |
| MysteryMechanism | Working out a hidden scientific mechanism from sparse observations. Run by Vals AI. | GPT-6 Astra53.1% | 21 | – | Vals AI |
| SciPredict | Predicting the outcomes of scientific experiments before they are run. Scale AI. | Gemini 3 Pro25.3% | 15 | – | Scale AI SEAL |
| Chess Puzzles | Solving chess puzzles from a board description. Run by Epoch AI. | GPT-6 Astra72.0% | 219 | – | Epoch AI Benchmarking Hub |
| Mystery Game Puzzles | Deduction puzzles in the style of mystery games. Run by Epoch AI. | GPT-6 Astra84.0% | 135 | – | Epoch AI Benchmarking Hub |
| CL-bench | Continual-learning style tasks that require using information given in context. | GPT-5.427.9% | 21 | – | Epoch AI Benchmarking Hub |
| CL-bench Life | The everyday-life subset of CL-bench. | GPT-5.522.2% | 16 | – | Epoch AI Benchmarking Hub |
| LMCA | A capability assessment with a published score ceiling. | Claude Opus 5.568.2% | 172 | – | Epoch AI Benchmarking Hub |
| DTBench | Decision-theory dilemmas scored against expert answers. | Claude Opus 5.598.9% | 207 | – | Epoch AI Benchmarking Hub |
| ForecastBench | Forecasting real future events, scored as they resolve. | o362.5% | 74 | – | ForecastBench |
| BTF-3 | Forecasting tournament pooled score. | GLM 5.3 Flash14.9% | 13 | – | Epoch AI Benchmarking Hub |
| HellaSwag | Commonsense sentence completion. A 2019 benchmark, saturated for current models; kept for history. | GPT 4 031495.3% | 68 | – | Epoch AI Benchmarking Hub |
| BIG-Bench Hard | Twenty-three hard tasks from BIG-Bench. A 2022 benchmark, saturated for current models; kept for history. | Gemini 1.5 Pro 00189.2% | 43 | – | Epoch AI Benchmarking Hub |
| WinoGrande | Pronoun-resolution commonsense questions. A 2019 benchmark, saturated for current models; kept for history. | Llama 3.1 405B89.2% | 74 | – | Epoch AI Benchmarking Hub |
| ARC (AI2 Challenge) | Grade-school science questions, challenge set. A 2018 benchmark, saturated for current models; kept for history. | DeepSeek V395.3% | 77 | – | Epoch AI Benchmarking Hub |
| PIQA | Physical commonsense questions. A 2019 benchmark, saturated for current models; kept for history. | GPT-4o mini88.7% | 56 | – | Epoch AI Benchmarking Hub |
| SuperGLUE | A suite of language-understanding tasks. A 2019 benchmark, saturated for current models; kept for history. | T5-11B88.9% | 8 | – | Epoch AI Benchmarking Hub |
| ANLI | Adversarially written natural-language inference. A 2019 benchmark, saturated for current models; kept for history. | GPT 3.5 Turbo 110658.1% | 11 | – | Epoch AI Benchmarking Hub |
| CommonsenseQA 2 | Commonsense yes/no questions. A 2022 benchmark, saturated for current models; kept for history. | T5-11B67.8% | 6 | – | Epoch AI Benchmarking Hub |
Coding
| Benchmark | What it measures | Leader | Models | Weight | Source |
|---|---|---|---|---|---|
| SWE-bench Verified (Epoch) | Epoch AI's own SWE-bench Verified runs with a constant scaffold. | Claude Opus 4.783.5% | 33 | 0.5 | Epoch AI Benchmarking Hub |
| SciCode | Research-grade scientific coding problems. | Claude Opus 5.566.9% | 209 | 0.5 | SciCode |
| WeirdML | Unusual machine-learning tasks that require writing working training code. | GPT-6 Astra Pro93.6% | 158 | 0.5 | WeirdML |
| FrontierCode | Hard real-world coding tasks run in agent harnesses (Cognition). | Claude Opus 5.554.6% | 41 | 0.5 | Cognition |
| GSO-Bench | Software optimisation tasks measured against expert speed-ups. | Claude Fable 5.188.2% | 35 | 0.5 | GSO-Bench |
| LMArena Coding | Text arena rating on coding prompts. | Gemini 4 Argon1560 | 373 | 0.5 | LMArena |
| LMArena WebDev | Pairwise votes on generated web apps. | Claude Opus 5.51815 | 128 | 1.0 | LMArena |
| LiveBench Coding | LiveBench coding category. | Claude Sonnet 5.591.4% | 63 | – | LiveBench |
| SciCode (AA) | Artificial Analysis' own SciCode run. Shown for reference. | Claude Opus 5.566.9% | 213 | – | Artificial Analysis |
| LiveCodeBench | Competitive programming problems released after model cutoffs. Run by Vals AI. | Claude Fable 5.190.5% | 137 | 1.0 | Vals AI |
| IOI | International Olympiad in Informatics problems. Run by Vals AI. | GPT-6 Astra100.0% | 36 | 0.5 | Vals AI |
| SWE-bench (Vals) | Vals AI's own SWE-bench run. Shown for reference. | Claude Opus 597.0% | 85 | – | Vals AI |
| SWE-Bench Pro | Long-horizon software engineering tasks in public repositories. Scale AI. | Muse Spark 1.161.5% | 24 | 1.0 | Scale AI SEAL |
| Aider Polyglot | Percent of 225 Exercism problems solved on the second attempt. | GPT-588.0% | 46 | 1.0 | Aider polyglot leaderboard |
| SWE-bench Verified (bash only) | Official SWE-bench Verified results from the minimal bash-only mini-SWE-agent harness. | Claude Opus 4.576.8% | 45 | 1.0 | SWE-bench |
| SWE-bench Verified (any scaffold) | Best official SWE-bench Verified result for the model with any agent scaffold. Shown for reference. | Claude Opus 4.576.8% | 47 | – | SWE-bench |
| Code Migration | Rewriting whole programs in another language so they still pass the tests. Run by Vals AI. | Claude Sonnet 5.569.8% | 67 | – | Vals AI |
| ProgramBench | Rebuilding programs from compiled binaries. Run by Vals AI. | Claude Opus 5.518.5% | 51 | – | Vals AI |
| Vibe Code Bench 1-100 | Extending a working web app, request by request, one hundred times. Run by Vals AI. | Claude Opus 5.530.4% | 19 | – | Vals AI |
| Vibe Code Bench v1.1 | Building web apps from scratch from a brief. Run by Vals AI. | Claude Sonnet 5.592.4% | 102 | – | Vals AI |
| SWE-Bench Pro (private) | The held-out commercial-repository set of SWE-Bench Pro. Scale AI. | Muse Spark 1.161.5% | 24 | – | Scale AI SEAL |
| SWE Atlas: Codebase QnA | Answering questions about a large codebase. Scale AI. | Claude Opus 563.2% | 22 | – | Scale AI SEAL |
| SWE Atlas: Refactoring | Refactoring tasks in real repositories. Scale AI. | GPT-6 Astra59.0% | 17 | – | Scale AI SEAL |
| SWE Atlas: Test Writing | Writing tests for existing code. Scale AI. | Claude Fable 5.167.0% | 22 | – | Scale AI SEAL |
| MirrorCode | Reproducing a program's behaviour from its outputs. Run by Epoch AI. | Claude Opus 5.577.4% | 9 | – | Epoch AI Benchmarking Hub |
| CadEval | Generating CAD models from specifications. | o374.0% | 15 | – | Epoch AI Benchmarking Hub |
| Surface Evolver Bench | Scientific computing with the Surface Evolver package. | Claude Fable 595.0% | 28 | – | Epoch AI Benchmarking Hub |
| DeepSWE | Software-engineering tasks in an agent harness with reasoning effort recorded. | GPT-6 Astra74.1% | 69 | – | Epoch AI Benchmarking Hub |
| CursorBench | Cursor's internal coding-agent evaluation, with reasoning level and cost. | Claude Opus 5.557.8% | 63 | – | Cursor |
| ALE-Bench | Long-horizon algorithm engineering on AtCoder heuristic contests. | GPT-6 Astra2951.3 | 119 | – | Epoch AI Benchmarking Hub |
| AlgoTune | Speeding up numeric code beyond reference implementations. | GPT-5.22.0 | 18 | – | AlgoTune |
| FrontierSWE | Frontier-difficulty software-engineering tasks across implementation, performance and research. | GPT-6 Astra65.5% | 17 | – | Epoch AI Benchmarking Hub |
| Terminal-Bench 4.0 (AA) | Real command-line tasks in a sandboxed terminal, version 4.0. Run by Artificial Analysis; the Terminal-Bench team's own run is the one in the index. | Claude Sonnet 5.563.6% | 201 | – | Artificial Analysis |
| Terminal-Bench 2.1 (AA) | The previous Terminal-Bench release, kept because far more models have been run on it than on 4.0. | Claude Fable 5.191.4% | 232 | – | Artificial Analysis |
Agents & tools
| Benchmark | What it measures | Leader | Models | Weight | Source |
|---|---|---|---|---|---|
| Terminal-Bench | Command-line agent tasks; best result per model across harnesses. | GPT-5.584.7% | 75 | 1.0 | Terminal-Bench |
| OSWorld-Verified 2.0 | Computer-use tasks in a real desktop environment. | Claude Opus 531.4% | 14 | 0.5 | OSWorld |
| GDPval | Win rate against industry professionals on economically valuable tasks (OpenAI). | GPT-5.249.7% | 11 | 0.5 | OpenAI |
| Cybench | Unguided capture-the-flag cybersecurity tasks. | Claude Opus 4.693.0% | 21 | 0.5 | Cybench |
| Remote Labor Index | Real freelance projects completed end to end (Scale AI / CAIS). | GPT-6 Astra20.8% | 15 | 0.5 | Scale AI / CAIS |
| APEX-Agents | Long-horizon professional agent tasks (Mercor). | Gemini 4 Argon82.2% | 50 | 0.5 | Mercor |
| LMArena Agent | Task-outcome score (0–100) from real agentic sessions judged by their users. | Claude Fable 5.114.3 | 47 | 0.5 | LMArena |
| LiveBench Agentic Coding | LiveBench agentic coding category. | DeepSeek V4.1 Flash77.3% | 63 | – | LiveBench |
| Terminal-Bench Hard | Hard subset of Terminal-Bench in a fixed harness. Run by Artificial Analysis. | GPT-5.6 Sol65.9% | 399 | 0.5 | Artificial Analysis |
| τ²-Bench Telecom (AA) | Artificial Analysis' τ²-Bench telecom run. Shown for reference. | GLM 5.299.1% | 405 | – | Artificial Analysis |
| Terminal-Bench 2.1 (Vals) | Terminal-Bench 2.1 in Vals AI's fixed harness. | Claude Opus 5.587.6% | 73 | 0.5 | Vals AI |
| MCP Atlas | Real-world tool use through the Model Context Protocol. Scale AI. | Muse Spark 1.188.1% | 33 | 1.0 | Scale AI SEAL |
| HiL-Bench | Whether agents notice information gaps and ask clarifying questions. Scale AI. | Claude Fable 5.161.5% | 17 | 0.5 | Scale AI SEAL |
| BFCL Overall | Berkeley Function Calling Leaderboard overall accuracy. | Claude Opus 4.577.5% | 83 | 1.0 | Berkeley Function Calling Leaderboard |
| τ²-bench | Mean pass^1 across the airline, retail and telecom domains. | Qwen3.5 397B A17B87.9% | 8 | 1.0 | τ²-bench |
| GDPval (AA) | Artificial Analysis' normalised run of OpenAI's GDPval professional-task comparison. | Claude Opus 5.567.3% | 271 | – | Artificial Analysis |
| τ²-Bench Banking (AA) | Artificial Analysis' τ²-Bench banking-domain run: a support agent using tools with a simulated customer. | Qwen3.8 Max51.3% | 201 | 1.0 | Artificial Analysis |
| ITBench SRE (AA) | Site-reliability incident tasks in a live IT environment, run by Artificial Analysis. | GPT-5.6 Sol56.2% | 45 | – | Artificial Analysis |
| Analyst Agent (AA) | Multi-step financial-analyst tasks with tools, run by Artificial Analysis. | Gemini 3.7 Flash60.0% | 37 | – | Artificial Analysis |
| APEX-Agents (AA) | Artificial Analysis' run of Mercor's long-horizon professional agent tasks. | Gemini 3.5 Flash47.0% | 33 | – | Artificial Analysis |
| CUA-bench | Playing six commercial video games with only a keyboard and a mouse; half are held out. Run by Vals AI. | GPT-6 Astra19.2% | 8 | – | Vals AI |
| CyberBench | Finding and patching security bugs in real code. Run by Vals AI. | GPT-6 Sol78.0% | 41 | – | Vals AI |
| Finance Agent v2 | Core financial-analyst tasks with tools. Run by Vals AI. | Gemini 4 Argon65.4% | 69 | – | Vals AI |
| Harvey's Legal Agent Benchmark | Agentic legal work over files, written with Harvey. Run by Vals AI. | Muse Spark 1.225.4% | 69 | – | Vals AI |
| Legal Research Bench | Agentic US legal research questions. Run by Vals AI. | Muse Spark 1.355.3% | 68 | – | Vals AI |
| Poker Agent | AI agents playing poker against each other. Run by Vals AI. | GPT-5.21131.8% | 16 | – | Vals AI |
| SkillsBench | How much reusable skills help an agent complete tasks. Run by Vals AI. | DeepSeek V4.1 Flash69.8% | 33 | – | Vals AI |
| SREBench | Site-reliability incidents to diagnose and fix. Run by Vals AI. | GPT-6 Astra56.9% | 15 | – | Vals AI |
| Tax Agent Bench | Agentic US corporate tax research. Run by Vals AI. | Claude Fable 5.177.6% | 60 | – | Vals AI |
| Terminal-Bench 2.0 (Vals) | Terminal-Bench 2.0 in Vals AI's fixed harness. Run by Vals AI. | GPT-5.573.2% | 64 | – | Vals AI |
| Terminal-Bench 4.0 (Vals) | Terminal-Bench 4.0, the frontier-difficulty set, in Vals AI's harness. Run by Vals AI. | Claude Opus 5.565.2% | 39 | – | Vals AI |
| Terminal-Bench Science | Expert-authored scientific research workflows in a terminal. Run by Vals AI. | GPT-6 Astra62.9% | 33 | – | Vals AI |
| Time Horizon Index: KSP | Running a Kerbal Space Program mission end to end. Run by Vals AI. | GPT-6 Astra90.5% | 11 | – | Vals AI |
| Web Search Index | Answering questions that need live web search, with native and independent tools. Run by Vals AI. | Claude Fable 5 Exa48.5% | 8 | – | Vals AI |
| LMArena Search | Pairwise votes on answers that used web search. | GPT-5.6 Sol1256 | 33 | – | LMArena |
| BALROG | Progress through classic text and grid games such as NetHack. | GPT-6 Astra68.3% | 37 | – | BALROG |
| DeepResearch Bench | Long-form research reports judged for depth and accuracy. | Claude Opus 4.655.3% | 40 | – | DeepResearch Bench |
| TheAgentCompany | Office tasks in a simulated software company. | DeepSeek V3.242.9% | 14 | – | TheAgentCompany |
| PostTrainBench | Post-training a small model to a target, end to end. | Claude Fable 541.8% | 12 | – | Epoch AI Benchmarking Hub |
| ExploitBench | Developing working exploits for known vulnerabilities. | Claude Mythos Preview73.8% | 9 | – | Epoch AI Benchmarking Hub |
| METR Time Horizons | METR's task suite behind the 50%-time-horizon measure. | Claude Mythos Preview85.2% | 42 | – | METR |
| GBAEval | Playing Game Boy Advance games as an agent. | Claude Opus 579.6% | 23 | – | Epoch AI Benchmarking Hub |
| Vending-Bench 2 | Running a simulated vending-machine business for a year. | GPT-6 Astra15514.7 | 66 | – | Andon Labs |
| AutomationBench | Office automation tasks run end to end. Scored on partial credit, so a model that gets most of a workflow right still registers. | Gemini 4 Argon77.5% | 51 | – | Artificial Analysis |
| GDP.pdf | Document work on real PDFs, scored all-or-nothing: every required element has to be right for the task to pass, which is why the scores look low. | GPT-6 Astra32.2% | 51 | – | Artificial Analysis |
| Harvey LAB | Legal work drawn from practice, built with Harvey. Run by Artificial Analysis. | Muse Spark 1.395.5% | 18 | – | Harvey |
| EnterpriseOps-Gym | Enterprise operations tasks in a simulated company: tickets, approvals and systems a model has to drive to finish a job. | Claude Fable 551.1% | 18 | – | Artificial Analysis |
| AA-Briefcase | Knowledge work delivered as real files — spreadsheets, decks, documents — graded head to head, so the score is an Elo rather than a percentage. | Claude Sonnet 5.51823 | 49 | – | Artificial Analysis |
| BrowseComp-Plus | BrowseComp questions answered against a fixed 100k-document corpus instead of the live web, so the retriever and the model can be judged apart. Scores shown are on the BM25 baseline retriever, the one column nearly every submission runs. Model coverage stops in late 2025. | GPT-557.6% | 21 | – | BrowseComp-Plus |
| BrowseComp | Hard questions whose answers are buried deep on the open web: the model has to browse, not recall. Published by OpenAI. | Gemini 3.1 Pro31.2% | 39 | – | OpenAI |
| DeepSearchQA | Research questions with many separate facts to gather; scored on answers that are fully correct, so missing one item fails the task. Published by Google. | Gemini Deep Research Agent66.1% | 12 | – | |
| FACTS Search | Using web search as a tool and synthesising what comes back without inventing anything. Published by Google. | GPT-6 Astra89.9% | 19 | – |
Maths
| Benchmark | What it measures | Leader | Models | Weight | Source |
|---|---|---|---|---|---|
| FrontierMath Tiers 1–3 | Unpublished research-level maths problems, tiers 1–3 (v2 private set). Run by Epoch AI. | GPT-6 Astra93.7% | 111 | 1.0 | Epoch AI Benchmarking Hub |
| FrontierMath Tier 4 | The hardest FrontierMath tier: problems that take expert mathematicians days. Run by Epoch AI. | GPT-6.1 Sol100.0% | 67 | 0.5 | Epoch AI Benchmarking Hub |
| OTIS Mock AIME | Mock AIME competition problems from 2024–2025. Run by Epoch AI. | GPT-6 Astra100.0% | 266 | 0.5 | Epoch AI Benchmarking Hub |
| MATH Level 5 | Hardest tier of the MATH dataset. Largely saturated, kept at half weight. Run by Epoch AI. | GPT-598.1% | 95 | 0.5 | Epoch AI Benchmarking Hub |
| ProofBench | Full written proofs graded for rigour (Vals AI). | Claude Opus 5.5100.0% | 75 | 0.5 | Vals AI |
| LiveBench Mathematics | LiveBench mathematics category. | Claude Opus 5.597.1% | 63 | – | LiveBench |
| AIME (Vals) | Recent AIME problems, run by Vals AI at stated reasoning effort. | Gemini 3.1 Pro98.1% | 92 | 0.5 | Vals AI |
| MGSM | Grade-school maths in ten languages. Run by Vals AI. | Claude Opus 4.595.2% | 71 | 0.5 | Vals AI |
| AIME 2026 | AIME I and II 2026, evaluated on release by MathArena. | GPT-5.5100.0% | 31 | 1.0 | MathArena |
| HMMT February 2026 | Harvard-MIT Mathematics Tournament, February 2026. MathArena. | GPT-5.598.5% | 31 | 1.0 | MathArena |
| IMO 2025 | International Mathematical Olympiad 2025, proof-graded. MathArena. | GPT-538.1% | 6 | 0.5 | MathArena |
| MathArena Apex | The hardest problems from recent competitions. MathArena. | Claude Opus 4.881.3% | 46 | 0.5 | MathArena |
| Omni-MATH (HELM) | Olympiad-level maths problems. Run by Stanford CRFM's HELM. | GPT-5 mini72.2% | 68 | 1.0 | HELM Capabilities |
| MATH 500 | Five hundred competition problems in algebra, probability and trigonometry. Run by Vals AI. | Gemini 3 Pro96.4% | 57 | – | Vals AI |
| LMArena Maths | Text arena rating on maths prompts. | Gemini 4 Argon1530 | 364 | 1.0 | LMArena |
| GSM8K | Grade-school maths word problems. A 2021 benchmark, saturated for current models; kept for history. | Deepseek Coder v294.5% | 73 | – | Epoch AI Benchmarking Hub |
Knowledge
| Benchmark | What it measures | Leader | Models | Weight | Source |
|---|---|---|---|---|---|
| SimpleQA Verified | Short factual questions; measures recall without hallucination. Run by Epoch AI. | GPT-6 Astra75.6% | 81 | 1.0 | Epoch AI Benchmarking Hub |
| LiveBench Data Analysis | LiveBench data analysis category. | GPT-6 Astra83.0% | 63 | – | LiveBench |
| AA-Omniscience | Knowledge recall that penalises hallucination: correct answers minus confident wrong ones, from −100 to 100. Run by Artificial Analysis. | Claude Opus 5.546.4 | 526 | 1.0 | Artificial Analysis |
| MMLU-Pro | Harder, reasoning-heavy successor to MMLU. Run by Vals AI. | Claude Fable 5.192.4% | 130 | 1.0 | Vals AI |
| LegalBench | Legal reasoning tasks written by lawyers. Run by Vals AI. | Claude Fable 588.6% | 141 | 0.5 | Vals AI |
| CorpFin | Questions over real corporate finance documents. Run by Vals AI. | Claude Opus 573.2% | 124 | 0.5 | Vals AI |
| TaxEval | US tax questions checked by professionals. Run by Vals AI. | Muse Spark 1.280.4% | 138 | 0.5 | Vals AI |
| MedQA | US medical licensing exam questions. Run by Vals AI. | o196.5% | 91 | 0.5 | Vals AI |
| PRBench Finance | Professional reasoning in finance, graded by practitioners. Scale AI. | Muse Spark 1.359.5% | 34 | 0.5 | Scale AI SEAL |
| PRBench Legal | Professional reasoning in legal practice, graded by practitioners. Scale AI. | Muse Spark 1.361.6% | 38 | 0.5 | Scale AI SEAL |
| MultiNRC | Native multilingual reasoning across languages. Scale AI. | Muse Spark 1.165.6% | 43 | 0.5 | Scale AI SEAL |
| MMLU-Pro (HELM) | Stanford CRFM's own MMLU-Pro run. Shown for reference; Vals AI's run is in the index. | Gemini 3 Pro90.3% | 68 | – | HELM Capabilities |
| BioMysteryBench | End-to-end bioinformatics investigations from raw data to a conclusion. Run by Vals AI. | Claude Sonnet 5.581.1% | 20 | – | Vals AI |
| CaseLaw v2 | Questions over Canadian court cases. Run by Vals AI. | Grok 4.379.3% | 52 | – | Vals AI |
| Excel Modeling Benchmark | Building financial models in Excel from a brief. Run by Vals AI. | Claude Fable 5.176.7% | 65 | – | Vals AI |
| MedCode | Assigning medical billing codes from clinical notes. Run by Vals AI. | Claude Opus 563.6% | 100 | – | Vals AI |
| MedScribe | Doctor administrative paperwork from encounter notes. Run by Vals AI. | Claude Opus 5.591.4% | 102 | – | Vals AI |
| Public Benefits Bench | Navigating SNAP benefits questions for applicants. Run by Vals AI. | Claude Opus 576.9% | 43 | – | Vals AI |
| DrugDiscoveryBench | Drug-discovery reasoning tasks graded by practitioners. Scale AI. | GPT-6 Astra68.7% | 18 | – | Scale AI SEAL |
| EBR-bench | Evidence-based reasoning over research literature. Run by Epoch AI. | GPT-6 Astra76.2% | 24 | – | Epoch AI Benchmarking Hub |
| GDP.pdf | Professional document tasks over PDFs. | GPT-6 Astra34.2% | 52 | – | Epoch AI Benchmarking Hub |
| MMLU | Fifty-seven-subject multiple-choice exam of world knowledge. A 2020 benchmark, saturated for current models; kept for history. | GPT-4o88.1% | 120 | – | Epoch AI Benchmarking Hub |
| TriviaQA | Trivia questions with evidence documents. A 2017 benchmark, saturated for current models; kept for history. | Llama 2-70B87.6% | 36 | – | Epoch AI Benchmarking Hub |
| OpenBookQA | Elementary science questions with an open book of facts. A 2018 benchmark, saturated for current models; kept for history. | Phi 3 Mini88.0% | 41 | – | Epoch AI Benchmarking Hub |
| LAMBADA | Predicting the last word of a passage. A 2016 benchmark, saturated for current models; kept for history. | Megatron-Turing NLG 530B87.2% | 24 | – | Epoch AI Benchmarking Hub |
| AA-Omniscience: accuracy | The share of AA-Omniscience questions answered correctly, before any penalty for confident wrong answers. | Claude Fable 5.167.2% | 526 | – | Artificial Analysis |
| AA-Omniscience: non-hallucination | How often a model declines to answer rather than inventing one, on questions it does not know. The counterweight to the accuracy score. | MiniCPM5-1B99.1% | 526 | – | Artificial Analysis |
Instruction following
| Benchmark | What it measures | Leader | Models | Weight | Source |
|---|---|---|---|---|---|
| LiveBench Language | LiveBench language category. | Claude Fable 590.7% | 63 | – | LiveBench |
| LiveBench Instruction Following | LiveBench instruction-following category. | Gemini 3.8 Flash81.4% | 63 | – | LiveBench |
| IFBench | Precise instruction following on unseen constraints. Run by Artificial Analysis. | Grok 4.383.3% | 414 | 1.0 | Artificial Analysis |
| MultiChallenge | Multi-turn instruction following under realistic conversation constraints. Scale AI. | Muse Spark75.5% | 29 | 1.0 | Scale AI SEAL |
| TutorBench | Tutoring tasks for high-school and AP subjects. Scale AI. | – | 0 | 0.5 | Scale AI SEAL |
| IFEval (HELM) | Strict instruction following on verifiable constraints. Run by Stanford CRFM's HELM. | Grok 3 mini95.1% | 68 | 1.0 | HELM Capabilities |
| WildBench (HELM) | Real user tasks graded by a rubric-guided judge. Run by Stanford CRFM's HELM. | Qwen3 235B A22B 250786.6% | 68 | 0.5 | HELM Capabilities |
| LMArena Instruction Following | Text arena rating on prompts with explicit instructions. | Gemini 4 Argon1529 | 378 | – | LMArena |
Human preference
| Benchmark | What it measures | Leader | Models | Weight | Source |
|---|---|---|---|---|---|
| LMArena Text | Overall text arena rating with style control. | Gemini 4 Argon1525 | 378 | 1.0 | LMArena |
| EQ-Bench 4 | Emotional intelligence in multi-turn role-play, judged pairwise into an Elo-style rating. | Claude Opus 51385 | 27 | 0.5 | EQ-Bench |
| LMArena Creative Writing | Text arena rating on creative-writing prompts. | Gemini 4 Argon1519 | 376 | – | LMArena |
| LMArena Multi-turn | Text arena rating on conversations with more than one turn. | Gemini 4 Argon1554 | 376 | – | LMArena |
| LMArena Longer Queries | Text arena rating on long prompts. | Gemini 4 Argon1545 | 356 | – | LMArena |
| Lech Mazur Creative Writing | Short-story writing judged by a panel of models. | GPT-586.0% | 42 | – | Lech Mazur |
Multimodal
| Benchmark | What it measures | Leader | Models | Weight | Source |
|---|---|---|---|---|---|
| LMArena Vision | Pairwise votes on image-understanding prompts. | Claude Fable 51325 | 153 | 1.0 | LMArena |
| MMMU-Pro | College-level multimodal understanding. Run by Artificial Analysis. | Claude Opus 5.587.7% | 275 | 1.0 | Artificial Analysis |
| VISTA | Vision-language understanding. Scale AI. | Gemini 2.5 Pro54.6% | 57 | 1.0 | Scale AI SEAL |
| MMMU (validation) | Official MMMU validation-set leaderboard. | GPT-5.185.4% | 49 | 0.5 | MMMU |
| MMMU-Pro (official) | Official MMMU-Pro leaderboard. Shown for reference. | Gemini 3 Pro81.0% | 28 | – | MMMU |
| MMMU-Pro (Vals) | Vals AI's own run of MMMU-Pro. Run by Vals AI. | Claude Fable 5.190.6% | 90 | – | Vals AI |
| MortgageTax | Reading tax certificate images and extracting the figures. Run by Vals AI. | Claude Opus 572.1% | 93 | – | Vals AI |
| SAGE | Grading handwritten maths work from images. Run by Vals AI. | Claude Opus 4.756.1% | 88 | – | Vals AI |
| VoiceCodeBench | Speech recognition of entities across eight workflow domains. Run by Vals AI. | GPT Live Transcribe67.7% | 20 | – | Vals AI |
| AudioMC | Multiple-choice questions asked and answered in audio. Scale AI. | Gemini 3.8 Flash60.4% | 27 | – | Scale AI SEAL |
| AudioMC (audio output) | AudioMC answered in speech. Scale AI. | GPT Realtime 248.5% | 15 | – | Scale AI SEAL |
| AudioMC (text output) | AudioMC answered in text. Scale AI. | Inkling Small54.9% | 17 | – | Scale AI SEAL |
| VTB | Scale's vision task benchmark across practical image tasks. Scale AI. | Muse Spark 1.144.8% | 21 | – | Scale AI SEAL |
| GeoBench | Identifying where a photo was taken. | Gemini 3 Flash88.0% | 27 | – | GeoBench |
| VPCT | Visual physics comprehension: predicting what happens next in a scene. | Gemini 3 Pro91.0% | 30 | – | Epoch AI Benchmarking Hub |
| Video-MME | Video understanding across durations, without subtitles. | video-SALMONN 2+79.7% | 42 | – | Video-MME |
| Blueprint-Bench 2 | Turning photos of rooms into floor plans. | Gemini 4 Argon54.5% | 31 | – | Epoch AI Benchmarking Hub |
| MindCube | Spatial reasoning from multiple views. | Gemma 3 12B46.7% | 5 | – | Epoch AI Benchmarking Hub |
| SpatialViz-Bench | Spatial visualisation: rotation, folding and animation. | Gemini 2.5 Pro44.7% | 8 | – | Epoch AI Benchmarking Hub |
| ScienceQA | Science questions with diagrams. A 2022 benchmark, saturated for current models; kept for history. | Phi 3.5 Vision91.3% | 23 | – | Epoch AI Benchmarking Hub |
Long context
| Benchmark | What it measures | Leader | Models | Weight | Source |
|---|---|---|---|---|---|
| Fiction.LiveBench 120k | Long-context comprehension of fiction at 120k tokens. | o3100.0% | 35 | 1.0 | Fiction.live |
| AA-LCR | Long-context reasoning across ~100k-token document sets. Run by Artificial Analysis. | Kimi K388.7% | 521 | 1.0 | Artificial Analysis |
| LMArena Document | Pairwise votes on answers about uploaded documents. | Claude Fable 51499 | 44 | – | LMArena |
| MLCR | Medical long-context reasoning: clinical questions answered over long patient records, scored on accuracy, completeness and concision together. | Claude Sonnet 5.575.0% | 32 | – | Artificial Analysis |
Composite
| Benchmark | What it measures | Leader | Models | Weight | Source |
|---|---|---|---|---|---|
| Epoch Capabilities Index | Epoch AI's own composite capability score fitted across many benchmarks. Shown for reference; not part of the BenchLeader Index. | Claude Opus 5.5167.3 | 241 | – | Epoch AI Benchmarking Hub |
| LiveBench | Average of LiveBench's category scores on the current question set. | Claude Fable 5.183.4% | 63 | 1.0 | LiveBench |
| AA Intelligence Index | Artificial Analysis' composite of its own evaluations (GPQA, HLE, IFBench, LCR, τ²-Bench, Terminal-Bench Hard, SciCode, AA-Omniscience and more). | Claude Opus 5.557.6 | 602 | 1.0 | Artificial Analysis |
| Vals Index | Vals AI's composite across its benchmarks. Shown for reference. | Gemini 4 Argon68.9 | 38 | – | Vals AI |
| HELM Capabilities mean | Mean of HELM's five capability scenarios. Shown for reference. | GPT-5 mini81.9% | 68 | – | HELM Capabilities |
| Vals Multimodal Index | Vals AI's GDP-weighted multimodal composite. Run by Vals AI. | Claude Fable 574.2% | 31 | – | Vals AI |
Safety & honesty
| Benchmark | What it measures | Leader | Models | Weight | Source |
|---|---|---|---|---|---|
| FORTRESS | Robustness to harmful requests without over-refusing benign ones. Scale AI. | DeepSeek R174.4% | 65 | – | Scale AI SEAL |
| MASK | Whether a model states what it believes when pressured to lie. Scale AI. | Claude Opus 4.696.3% | 64 | – | Scale AI SEAL |
| PropensityBench | How readily an agent takes harmful actions when pressured in agentic settings. Scale AI. | Gemini 2.5 Pro79.0% | 14 | – | Scale AI SEAL |