BenchLeader
Menu
Best LLMsChoose a modelLeaderboardsModelsBenchmarksProvidersReleases9

MedQA

US medical licensing exam questions. Run by Vals AI.

As of 25 Oct 2026, o1 leads MedQA on BenchLeader with 96.5%, ahead of GPT-5.1 at 96.4%, across 91 model configurations with a published result.

Published by
Vals AI
Category
Knowledge
Index weight
0.5
Models
91
Data as of
25 Oct 2026
Newest result
16 Apr 2026

The publisher has posted no new results on this board since 16 Apr 2026. BenchLeader re-reads it every day, so newer models appear here as soon as they are published.

Vals AI (vals.ai).

What the test looks like

MedQA is built from US medical licensing exam (USMLE) questions: clinical vignettes with multiple-choice answers.

How it is scored

Accuracy, run by Vals AI.

What to keep in mind

Multiple-choice exam performance is not clinical competence, and frontier models are near ceiling on it.

Top 15
  1. 1o1 (high)96.5%
  2. 2GPT-5.1 (high)96.4%
  3. 3Gemini 3.1 Pro (high)96.4%
  4. 4GPT-5 (high)96.3%
  5. 5GPT-5.4 (xhigh)96.1%
  6. 6GPT-5 mini (high)96.1%
  7. 7o3 (high)96.1%
  8. 8Gemini 3 Pro (high)96.0%
  9. 9o4-mini (high)96.0%
  10. 10Claude Opus 4.5 (thinking)95.9%
  11. 11Gemini 3 Flash (high)95.8%
  12. 12Claude Opus 4.6 (thinking)95.4%
  13. 13Qwen3.5 Plus (thinking)95.2%
  14. 14o3-mini (high)94.8%
  15. 15Claude Sonnet 4.5 (thinking)94.7%
91 of 91
#
1o1high96.5%48.32026-04-16
2GPT-5.1high96.4%57.72026-04-16
3Gemini 3.1 Prohigh96.4%60.32026-04-16
4GPT-5high96.3%57.92026-04-16
5GPT-5.4xhigh96.1%64.72026-04-16
6GPT-5 minihigh96.1%54.12026-04-16
7o3high96.1%53.92026-04-16
8Gemini 3 Prohigh96.0%60.02026-04-16
9o4-minihigh96.0%50.12026-04-16
10Claude Opus 4.5thinking95.9%59.92026-04-16
11Gemini 3 Flashhigh95.8%58.32026-04-16
12Claude Opus 4.6thinking95.4%62.52026-04-16
13Qwen3.5 Plusthinking95.2%57.32026-04-16
14o3-minihigh94.8%47.22026-04-16
15Claude Sonnet 4.5thinking94.7%52.62026-04-16
16Grok 4.20thinking94.5%58.62026-04-16
17Kimi K2.5thinking94.4%57.62026-04-16
18GLM-5thinking94.3%58.92026-04-16
19GPT-5.2xhigh94.1%60.72026-04-16
20DeepSeek V3.2high93.9%–2026-04-16
21GLM 4.793.7%51.92026-04-16
22Claude Opus 4.1thinking93.6%56.42026-04-16
23GPT-5 nanohigh93.3%47.42026-04-16
24Claude Opus 4.593.2%57.92026-04-16
25Gemini 2.5 Pro93.1%53.92026-04-16
26Claude Opus 492.9%52.62026-04-16
27Claude Sonnet 4thinking92.7%53.22026-04-16
28Kimi K2 Thinkingthinking92.6%52.42026-04-16
29MiniMax M2.592.5%53.62026-04-16
30Claude Opus 4.192.5%51.52026-04-16
31Grok 492.5%58.02026-04-16
32Grok 292.3%41.92026-04-16
33GLM-4.692.2%49.92026-04-16
34Grok 4.1 Fastthinking92.1%54.72026-04-16
35Grok 4 Fastthinking92.1%51.42026-04-16
36Claude Sonnet 4.692.1%57.12026-04-16
37Gemini 2.5 Flash 09 202591.4%46.92026-04-16
38gpt-oss-120b91.4%45.62026-04-16
39GPT-4.1high91.2%46.72026-04-16
40Gemini 2.5 Flash 09 2025thinking91.2%51.12026-04-16
41MiniMax-M2.191.2%53.82026-04-16
42Gemini 2.5 Flashthinking91.0%47.72026-04-16
43DeepSeek R190.8%46.82026-04-16
44Qwen3 235B A22B90.6%50.12026-04-16
45Claude Sonnet 490.3%50.52026-04-16
46Claude 3.7 Sonnetthinking90.2%51.02026-04-16
47o1-minihigh90.2%–2026-04-16
48Grok 3 Mini Fasthigh90.1%53.72026-04-16
49GLM-4.590.0%51.72026-04-16
50Magistral Medium 1.289.5%43.92026-04-16
51Gemini 2.5 Flash Lite 09thinking88.9%47.92026-04-16
52Grok 3 Mini Fastlow88.7%50.72026-04-16
53Llama Meta Llama 3.1 405B Turbo88.2%–2026-04-16
54GPT-4ohigh88.2%41.72026-04-16
55Qwen3 Maxmax87.4%52.82026-04-16
56Gemini 2.5 Flash86.7%52.72026-04-16
57Llama Meta Llama 3.1 70B Turbo84.8%–2026-04-16
58GPT-4.1 minihigh84.6%46.32026-04-16
59Kimi K284.0%50.42026-04-16
60Grok 383.8%51.52026-04-16

More knowledge benchmarks · All LLM benchmarks

Cite as: BenchLeader, “MedQA leaderboard”, data as of 25 Oct 2026, https://www.benchleader.com/benchmarks/vals_medqa.

MedQA: questions

What does MedQA measure?
MedQA is built from US medical licensing exam (USMLE) questions: clinical vignettes with multiple-choice answers. Scores are reported in percent of tasks solved; higher is better.
Which AI model leads MedQA?
o1 leads MedQA with 96.5% as of 25 Oct 2026, ahead of GPT-5.1 at 96.4%.
How many models have MedQA results?
91 model configurations have a MedQA result on BenchLeader, all taken from Vals AI.
Who runs MedQA and how often is it updated?
MedQA is published by Vals AI. BenchLeader re-reads the published results once a day and records the date each result was published.
Does MedQA count toward the BenchLeader Index?
Yes. MedQA contributes to the knowledge category of the BenchLeader Index, normalised so that 50 is the average of the evaluated models.