MMLU-Pro (HELM)
Stanford CRFM's own MMLU-Pro run. Shown for reference; Vals AI's run is in the index.
As of 12 Sept 2026, Gemini 3 Pro leads MMLU-Pro (HELM) on BenchLeader with 90.3%, ahead of Claude Opus 4 at 87.5%, across 68 model configurations with a published result.
- Published by
- HELM Capabilities
- Category
- Knowledge
- Index weight
- Reference only
- Models
- 68
- Data as of
- 12 Sept 2026
Stanford Center for Research on Foundation Models (crfm.stanford.edu), Apache-2.0 results.
What the test looks like
MMLU-Pro is a harder successor to MMLU: about 12,000 questions across 14 subjects with ten answer options each and more reasoning-heavy items, removing the trivial and noisy questions of the original.
How it is scored
Accuracy, run by Vals AI.
What to keep in mind
Stanford CRFM's own run of MMLU-Pro in HELM's fixed harness. Shown for reference; Vals AI's run is counted in the index. See also the MMLU-Pro board.
- 1Gemini 3 Pro90.3%
- 2Claude Opus 487.5%
- 3Claude Sonnet 4.586.9%
- 4GPT-586.3%
- 5Gemini 2.5 Pro86.3%
- 6o385.9%
- 7Claude Opus 485.9%
- 8Grok 485.1%
- 9Qwen3 235B A22B 250784.4%
- 10Claude Sonnet 484.3%
- 11Claude Sonnet 484.3%
- 12GPT-5 mini83.5%
- 13o4-mini82.0%
- 14Kimi K281.9%
- 15Qwen3 235B A22B81.7%
68 of 68
| # | |||||
|---|---|---|---|---|---|
| 1 |  Gemini 3 ProGoogle | 90.3% | – | 62.0 | – |
| 2 |  Claude Opus 4Anthropic | 87.5% | – | 56.1 | – |
| 3 | Claude Sonnet 4.5Anthropic | 86.9% | 20250929 | 53.7 | – |
| 4 |  GPT-5OpenAI | 86.3% | – | 60.8 | – |
| 5 | Gemini 2.5 ProGoogle | 86.3% | – | 54.5 | – |
| 6 | o3OpenAI | 85.9% | – | 61.6 | – |
| 7 | Claude Opus 4Anthropic | 85.9% | 20250514 | 52.2 | – |
| 8 |  Grok 4xAI | 85.1% | 0709 | 58.6 | – |
| 9 |  Qwen3 235B A22B 2507Alibabaopen | 84.4% | Qwen3 235B A22B Instruct 2507 FP8 | 54.1 | – |
| 10 | Claude Sonnet 4Anthropic | 84.3% | – | 55.1 | – |
| 11 | Claude Sonnet 4Anthropic | 84.3% | 20250514 | 50.5 | – |
| 12 | GPT-5 miniOpenAI | 83.5% | – | 57.9 | – |
| 13 | o4-miniOpenAI | 82.0% | – | 57.2 | – |
| 14 |  Kimi K2Moonshot AIopen | 81.9% | Kimi K2 Instruct | 51.2 | – |
| 15 | Qwen3 235B A22BAlibabaopen | 81.7% | Qwen3 235B A22B FP8 Throughput | 44.8 | – |
| 16 | GPT-4.1OpenAI | 81.1% | – | 48.8 | – |
| 17 |  Llama 4 MaverickMetaopen | 81.0% | Llama 4 Maverick (17Bx128E) Instruct FP8 | 44.2 | – |
| 18 |  Palmyra X5Writer | 80.4% | Palmyra X5 | – | – |
| 19 | Grok 3 minixAI | 79.9% | – | 51.8 | – |
| 20 | gpt-oss-120bOpenAIopen | 79.5% | – | 48.8 | – |
| 21 |  DeepSeek R1 0528DeepSeekopen | 79.3% | – | 49.9 | – |
| 22 | Grok 3xAI | 78.8% | – | 51.0 | – |
| 23 | Qwen3-Next 80B A3BAlibabaopen | 78.6% | – | 51.2 | – |
| 24 | Claude 3.7 SonnetAnthropic | 78.4% | 20250219 | 50.2 | – |
| 25 | GPT-4.1 miniOpenAI | 78.3% | – | 46.3 | – |
| 26 | GPT-5 nanoOpenAI | 77.8% | – | 52.5 | – |
| 27 | Claude Haiku 4.5Anthropic | 77.7% | 20251001 | 48.6 | – |
| 28 | Claude 3.5 SonnetAnthropic | 77.7% | 20241022 | 46.3 | – |
| 29 |  GLM-4.5-AirZhipu AIopen | 76.2% | – | 47.0 | – |
| 30 | Llama 4 ScoutMetaopen | 74.2% | Llama 4 Scout (17Bx16E) Instruct | 40.4 | – |
| 31 | gpt-oss-20bOpenAIopen | 74.0% | – | 46.0 | – |
| 32 | Gemini 1.5 Pro 002Google | 73.7% | Gemini 1.5 Pro (002) | 44.8 | – |
| 33 | Gemini 2.0 FlashGoogle | 73.7% | Gemini 2.0 Flash | 43.5 | – |
| 34 |  Nova PremierAmazon | 72.6% | – | 45.8 | – |
| 35 | DeepSeek V3DeepSeekopen | 72.3% | DeepSeek v3 | 45.2 | – |
| 36 | Llama 3.1 405BMetaopen | 72.3% | Llama 3.1 Instruct Turbo (405B) | 43.1 | – |
| 37 | Gemini 2.0 Flash-LiteGoogle | 72.0% | – | 47.7 | – |
| 38 | GPT-4oOpenAI | 71.3% | – | 43.8 | – |
| 39 | Gemini 1.5 Flash 002Google | 67.8% | Gemini 1.5 Flash (002) | 40.8 | – |
| 40 | Nova ProAmazon | 67.3% | – | 41.1 | – |
| 41 | Palmyra X 004Writer | 65.7% | – | – | – |
| 42 | Llama 3.1 70BMetaopen | 65.3% | Llama 3.1 Instruct Turbo (70B) | 41.1 | – |
| 43 | Gemini 2.5 FlashGoogle | 63.9% | – | 50.0 | – |
| 44 | Qwen2.5 72BAlibabaopen | 63.1% | Qwen2.5 Instruct Turbo (72B) | 42.7 | – |
| 45 |  Mistral Small 3.1Mistral AIopen | 61.0% | 2503 | 40.3 | – |
| 46 | Claude 3.5 HaikuAnthropic | 60.5% | 20241022 | 40.0 | – |
| 47 | GPT-4o miniOpenAI | 60.3% | – | 36.6 | – |
| 48 | Amazon Nova LiteAmazon | 60.0% | – | 39.4 | – |
| 49 | Mistral Large 2Mistral AIopen | 59.9% | 2411 | 40.8 | – |
| 50 | Palmyra FinWriter | 59.1% | – | – | – |
| 51 | GPT-5.1OpenAI | 57.9% | – | 59.4 | – |
| 52 |  Granite 4.0 SmallIBM | 56.9% | IBM Granite 4.0 Small | – | – |
| 53 | GPT-4.1 nanoOpenAI | 55.0% | – | 39.1 | – |
| 54 | Qwen2.5 7B InstructAlibabaopen | 53.9% | Qwen2.5 Instruct Turbo (7B) | 40.1 | – |
| 55 | Gemini 2.5 Flash-LiteGoogle | 53.7% | – | 43.8 | – |
| 56 | Nova MicroAmazon | 51.1% | – | 39.8 | – |
| 57 | Mixtral 8x22B InstructMistral AIopen | 46.0% | Mixtral Instruct (8x22B) | 37.6 | – |
| 58 |  Olmo 2 32B March 2025Ai2 | 41.4% | OLMo 2 32B Instruct March 2025 | – | – |
| 59 | Palmyra MedWriter | 41.1% | – | – | – |
| 60 | Llama 3.1 8BMetaopen | 40.6% | Llama 3.1 Instruct Turbo (8B) | 36.3 | – |
Cite as: BenchLeader, “MMLU-Pro (HELM) leaderboard”, https://www.benchleader.com/benchmarks/helm_mmlu_pro, data as of 12 Sept 2026.
MMLU-Pro (HELM): questions
- What does MMLU-Pro (HELM) measure?
- MMLU-Pro is a harder successor to MMLU: about 12,000 questions across 14 subjects with ten answer options each and more reasoning-heavy items, removing the trivial and noisy questions of the original. Scores are reported in percent of tasks solved; higher is better.
- Which AI model leads MMLU-Pro (HELM)?
- Gemini 3 Pro leads MMLU-Pro (HELM) with 90.3% as of 12 Sept 2026, ahead of Claude Opus 4 at 87.5%.
- How many models have MMLU-Pro (HELM) results?
- 68 model configurations have a MMLU-Pro (HELM) result on BenchLeader, all taken from HELM Capabilities.
- Who runs MMLU-Pro (HELM) and how often is it updated?
- MMLU-Pro (HELM) is published by HELM Capabilities. BenchLeader re-reads the published results every morning and records the date each result was published.
- Does MMLU-Pro (HELM) count toward the BenchLeader Index?
- No. MMLU-Pro (HELM) is shown for reference but left out of the composite index.