OpenAIReasoning model

o3

Name: o3 benchmark results
Creator: BenchLeader

Best configuration ranks #90 of 610 on the BenchLeader Index at 60.3 ±3.0. Last measured 2 Sept 2026. Released 16 Apr 2025.

Blended price: $3.50/M; $2.00 in · $8.00 out
Output speed: 139 tok/s
First answer: 6.03 s; first token 4.30 s
Context: 200k

How it scores by categoryDashed line = average model (50). One step of 15 = one standard deviation.

Overall index60
Reasoning61
Coding62
Agents & tools70
Knowledge58
Instruction following70
Human preference62
Multimodal58
Long context64
Composite54

Reasoning-effort configurations

The same model behaves differently depending on how much it is allowed to think. Each row is one setting, scored only on the benchmarks that were run at that setting. “Default” means the publisher did not say which setting was used.

Effort	Index	Rank	Speed	First answer	Chat reply cost	Categories
low	47.2	#354	139 tok/s	6.03 s	$0.0032	Maths 42 · Reasoning 49
medium	53.6	#217	139 tok/s	6.03 s	$0.0032	Agents & tools 47 · Coding 37 · Instruction following 54 · Knowledge 58 · Long context 80 · Maths 50 · Multimodal 63 · Reasoning 48
high	52.9	#235	139 tok/s	6.03 s	$0.0032	Agents & tools 42 · Coding 54 · Instruction following 51 · Knowledge 58 · Maths 54 · Multimodal 64 · Reasoning 51
defaultbest	60.3	#90	139 tok/s	6.03 s	$0.0032	Agents & tools 70 · Coding 62 · Composite 54 · Human preference 62 · Instruction following 70 · Knowledge 58 · Long context 64 · Multimodal 58 · Reasoning 61

Benchmark results

One column per reasoning effort. Rank is among every configuration of every model on that benchmark. Hover a score for the run it came from.

Reasoning

Benchmark	low	medium	high	default	Source
GPQA Diamond	79.8%#114	80.8%#110	81.8%#105	–	Epoch AI Benchmarking Hub
Humanity's Last Exam	–	19.2%#19	20.3%#17	–	Scale AI / CAIS
SimpleBench	–	–	53.1%#43	–	SimpleBench
LMArena Hard Prompts	–	–	–	1441#105	LMArena
GPQA Diamond (AA)not in index	–	–	–	82.7%#165	Artificial Analysis
Humanity's Last Exam (AA)not in index	–	–	–	20.1%#159	Artificial Analysis
GPQA Diamond (Vals)not in index	–	–	84.1%#52	–	Vals AI
Kagi LLM Benchmark	–	–	–	67.6%#32	Kagi LLM Benchmark
ARC-AGI-1	75.7%#91	53.8%#119	60.8%#107	–	ARC Prize
ARC-AGI-2	4.0%#127	3.0%#135	6.5%#113	–	ARC Prize

Coding

Benchmark	low	medium	high	default	Source
SWE-bench Verified (Epoch)	–	62.3%#28	–	–	Epoch AI Benchmarking Hub
WeirdML	–	–	52.4%#62	–	WeirdML
GSO-Bench	–	–	8.8%#18	–	GSO-Bench
LMArena Coding	–	–	–	1460#113	LMArena
LiveCodeBench	–	–	83.9%#43	–	Vals AI
Aider Polyglot	–	–	–	81.3%#4	Aider polyglot leaderboard
SWE-bench Verified (bash only)	–	–	–	58.4%#25	SWE-bench
SWE-bench Verified (any scaffold)not in index	–	–	–	58.4%#34	SWE-bench

Agents & tools

Benchmark	low	medium	high	default	Source
GDPval	–	30.8%#7	–	–	OpenAI
APEX-Agents	–	–	17.2%#43	–	Mercor
Terminal-Bench Hard	–	–	–	37.1%#67	Artificial Analysis
τ²-Bench Telecom (AA)not in index	–	–	–	80.7%#117	Artificial Analysis
BFCL Overall	–	–	–	63.0%#7	Berkeley Function Calling Leaderboard

Maths

Benchmark	low	medium	high	default	Source
FrontierMath Tiers 1–3	19.3%#75	29.8%#59	33.3%#56	–	Epoch AI Benchmarking Hub
OTIS Mock AIME	60.0%#146	84.4%#83	83.9%#90	–	Epoch AI Benchmarking Hub
MATH Level 5	–	–	97.8%#5	–	Epoch AI Benchmarking Hub
AIME (Vals)	–	–	85.3%#37	–	Vals AI
MGSM	–	–	91.8%#26	–	Vals AI
IMO 2025	–	–	16.7%#4	–	MathArena

Knowledge

Benchmark	low	medium	high	default	Source
SimpleQA Verified	–	–	49.4%#22	–	Epoch AI Benchmarking Hub
AA-Omniscience	–	–	–	-15.6#164	Artificial Analysis
MMLU-Pro	–	–	85.6%#54	–	Vals AI
LegalBench	–	–	83.8%#44	–	Vals AI
CorpFin	–	–	59.7%#72	–	Vals AI
TaxEval	–	–	74.6%#33	–	Vals AI
MedQA	–	–	96.1%#6	–	Vals AI
PRBench Finance	–	–	–	47.7%#12	Scale AI SEAL
PRBench Legal	–	–	–	48.6%#12	Scale AI SEAL
MultiNRC	–	44.5%#16	45.5%#14	–	Scale AI SEAL

Instruction following

Benchmark	low	medium	high	default	Source
IFBench	–	–	–	71.4%#56	Artificial Analysis
MultiChallenge	–	–	56.6%#16	–	Scale AI SEAL
TutorBench	–	52.8%#13	52.1%#14	–	Scale AI SEAL

Human preference

Benchmark	low	medium	high	default	Source
LMArena Text	–	–	–	1431#91	LMArena

Multimodal

Benchmark	low	medium	high	default	Source
LMArena Vision	–	–	–	1214#66	LMArena
MMMU-Pro	–	–	–	70.1%#116	Artificial Analysis
VISTA	–	49.6%#12	50.1%#10	–	Scale AI SEAL
MMMU (validation)	–	–	–	82.9%#3	MMMU
MMMU-Pro (official)not in index	–	–	–	76.4%#6	MMMU

Long context

Benchmark	low	medium	high	default	Source
Fiction.LiveBench 120k	–	100.0%#1	–	–	Fiction.live
AA-LCR	–	–	–	74.7%#114	Artificial Analysis

Composite

Benchmark	low	medium	high	default	Source
Epoch Capabilities Indexnot in index	–	–	–	146.9#56	Epoch AI Benchmarking Hub
AA Intelligence Index	–	–	–	20.2#173	Artificial Analysis

Where it wins

Benchmarks where this configuration ranks in the top five of every configuration measured.

Where to run it

Every provider serving this model through OpenRouter, with throughput and first-token latency measured on live traffic over the last 30 minutes and each provider’s own price. Purple marks the best in each column.

Provider	Speed	First token	Input $/M	Output $/M	Context	Quantisation
OpenAI	57 tok/s	4.30 s	$2.00	$8.00	200k	–

What a task costs

Estimates from list price, output speed and time to first answer for the best configuration. “With caching” assumes three-quarters of the input is served from the prompt cache at $0.500 per 1M. Reasoning tokens are not modelled.

Workload	Tokens in / out	Cost	With caching	Time
Chat reply	400 / 300	$0.0032	$0.0028	8.2 s
Summarise a 30-page report	12,000 / 600	$0.029	$0.015	10.4 s
Code edit	6,000 / 1,500	$0.024	$0.017	16.9 s
Agentic coding session	60,000 / 4,000	$0.152	$0.085	34.9 s
Structured extraction	2,000 / 200	$0.0056	$0.0034	7.5 s