BenchLeader

Agnes 2.5 Pro Beta vs GPT-6 Sol

Full profiles: Agnes 2.5 Pro Beta (Sapiens AI) and GPT-6 Sol (OpenAI): every benchmark, price and rank.

Updated · Data from 11 sources

As of 16 Nov 2026, there are not enough shared benchmarks for a verdict, and Agnes 2.5 Pro Beta is 27× cheaper at $0.150 vs $4.00 per 1M tokens blended. Choose Agnes 2.5 Pro Beta for agents & tools (64 vs 59); choose GPT-6 Sol for multimodal (61 vs 58).

Key facts (as of 16 Nov 2026)

  • Agnes 2.5 Pro Beta costs $0.150 per 1M tokens blended (list price, 16 Nov 2026).
  • GPT-6 Sol costs $4.00 per 1M tokens blended (list price, 16 Nov 2026).
  • Of the 7 benchmarks both models have, GPT-6 Sol leads 7 (BenchLeader, 16 Nov 2026).

Which to pick for…

The pick's numbers come first. Category scores on the BenchLeader scale (50 is the average model, 15 points one standard deviation). High confidence means a lead of half a standard deviation or more; under one point is called either.

Use casePickWhyConfidence
Agents & tool useAgnes 2.5 Pro Beta64 vs 59 category score; GDPval-AA v2.1 1335 vs 1514Medium
Long documentsEither66 vs 67 category score; AA-LCR v1.1 83.0% vs 83.7%Either
Images and other mediaGPT-6 Sol61 vs 58 category score; MMMU-Pro 83.0% vs 75.4%Medium
Chat quality (human preference)–Only GPT-6 Sol has an Arena rating (64 category score)Not comparable
High volume on a budgetNeither flagshipA smaller model is usually the better fit: Cheapest good LLMs, Fastest LLMs–

Key differences

  • Not enough shared benchmarks for a verdict.
  • Agnes 2.5 Pro Beta is stronger in agents & tools.
  • GPT-6 Sol (max) is stronger in long context, multimodal.
  • Agnes 2.5 Pro Beta is 27× cheaper ($0.150 vs $4.00 per 1M blended).

At a glance

SpecAgnes 2.5 Pro BetaGPT-6 Sol
DeveloperSapiens AIOpenAI
Released26 Aug 202622 Sep 2026
Context window (tokens)1,000,0001,050,000
Compared atAgnes 2.5 Pro BetaGPT-6 Sol (max)
Open weightsNoNo
Price in / out / cached input per 1M–$2.00 / $10.00 / $0.200

Category by category

Category scores on the BenchLeader scale: 50 is the average evaluated model, 15 points is one standard deviation. A gap under one point is called level.

CategoryAgnes 2.5 Pro BetaGPT-6 Sol (max)Leader
Reasoning–66.0not comparable
Coding–66.3not comparable
Agents & tools64.458.8Agnes 2.5 Pro Beta
Maths–68.0not comparable
Knowledge–60.6not comparable
Human preference–64.2not comparable
Multimodal57.760.6GPT-6 Sol
Long context66.266.6level

On one chart

Each domain on the index's own scale, where 50 is an average model. Click a domain to see the benchmarks behind it. Pick up to 5 models from the list.

2550 avg75Agents & toolsLong contextMultimodalAgnes 2.5 Pro Beta, Agents & tools: 64.4Agnes 2.5 Pro Beta, Long context: 66.2Agnes 2.5 Pro Beta, Multimodal: 57.7GPT-6 Sol, Agents & tools: 58.8GPT-6 Sol, Long context: 66.6GPT-6 Sol, Multimodal: 60.6

Not drawn, measured for fewer than two of these models: Reasoning, Coding, Maths, Knowledge, Instruction following, Human preference.

Model, click to add or removeIndex

Where sources disagree

Across the 7 benchmarks both have, Agnes 2.5 Pro Beta leads 0, GPT-6 Sol leads 7.

Each publisher runs its own tasks, harnesses and effort settings, so their overall verdicts can differ; BenchLeader reconciles them on one scale. † measured at another effort setting of the same model (the publisher did not test the one compared here).

Price, speed and latency

List prices blended at three input tokens per output token. Per-task prices use the token counts on the methodology page. Cost, tokens and time per benchmark task are measured: Artificial Analysis’ averages over its whole Intelligence Index run.

MeasureAgnes 2.5 Pro BetaGPT-6 Sol (max)
Blended price per 1M tokens–$4.00
Input / output per 1M– / –$2.00 / $10.00
Cached input per 1M–$0.200
Price per task: chat reply–$0.0038
Price per task: summarize a 30-page report–$0.030
Price per task: code edit–$0.027
Price per task: agentic coding session–$0.160
Price per task: structured extraction–$0.0060
Agentic coding session, input mostly cached–$0.079
Cost per benchmark task (AA Index)–$2.47
Output tokens per task (AA Index)–54k
Of which reasoning–35k
Time per task (AA Index)–7 min
Output speed–127 tok/s
Time to first answer–36.9 s
Time to a complete answer, last measured 13 Nov 2026–143.8 s

Each model is compared at its best-ranked reasoning-effort setting. Agnes 2.5 Pro Beta. GPT-6 Sol (max) scores 62.6 and takes 37 s to its first answer token; GPT-6 Sol (xhigh) scores 58.4 in 19 s, GPT-6 Sol (high) scores 58.1 in 6 s. Higher effort means more thinking before the answer: better scores, longer waits and more tokens billed. How reasoning effort changes scores and cost.

Every benchmark

120 benchmarks, grouped by category; each name links to the board and its leaders. † measured at another effort setting of the same model, shown for reference and not counted as a win.

MetricPublisherAgnes 2.5 Pro BetaGPT-6 Sol (max)
BenchLeader IndexBenchLeader–62.6
Agents & tools scoreBenchLeader64.458.8
Long context scoreBenchLeader66.266.6
Multimodal scoreBenchLeader57.760.6
Coding scoreBenchLeader–66.3
Human preference scoreBenchLeader–64.2
Knowledge scoreBenchLeader–60.6
Maths scoreBenchLeader–68.0
Reasoning scoreBenchLeader–66.0
Blended price $/MBenchLeader–$4.00
Output speedBenchLeader–127 tok/s
Time to first answerBenchLeader–36.9 s
Context windowBenchLeader1,000,000 tokens1,050,000 tokens
Reasoning
GPQA DiamondEpoch AI Benchmarking Hub–94.3%
SimpleBenchSimpleBench–73.1%†
Arena Hard PromptsArena (LMArena)–1483
LiveBench 1.0 ReasoningLiveBench–88.7%
GPQA Diamond (AA)Artificial Analysis90.5%–
Humanity's Last Exam (AA)Artificial Analysis37.5%47.9%
HLE-Diamond (AA)Artificial Analysis–45.2%
ARC-AGI-1ARC Prize–95.5%
ARC-AGI-2ARC Prize–89.6%
ARC-AGI-3ARC Prize–23.0%
CritPtArtificial Analysis15.7%30.9%
CritPt v2 hard subset (AA)Artificial Analysis–66.5%
MysteryMechanismVals AI–30.2%
HLE-Diamond (Scale)Scale AI SEAL–33.8%†
Arena ExpertArena (LMArena)–1496
Chess PuzzlesEpoch AI Benchmarking Hub–56.0%
Mystery Game PuzzlesEpoch AI Benchmarking Hub–56.0%
LMCAEpoch AI Benchmarking Hub–59.1%
DTBenchEpoch AI Benchmarking Hub–97.3%
Coding
SciCodeSciCode–57.6%
FrontierCodeCognition–49.3%
Arena CodingArena (LMArena)–1521
Arena WebDevArena (LMArena)–1673
LiveBench 1.0 CodingLiveBench–81.8%
SciCode (AA)Artificial Analysis48.8%57.6%
Show all 120 benchmarks
MetricPublisherAgnes 2.5 Pro BetaGPT-6 Sol (max)
Coding
IOIVals AI–82.6%
Code MigrationVals AI–57.2%
ProgramBenchVals AI–2.0%
Vibe Code Bench v1.1Vals AI–87.8%
MirrorCodeEpoch AI Benchmarking Hub–48.9%
ALE-BenchEpoch AI Benchmarking Hub–2462
WeirdML v3WeirdML–19.7%†
Terminal-Bench 4.0 (AA)Artificial Analysis–43.9%
Terminal-Bench 2.1 (AA)Artificial Analysis69.7%–
GameBench 2 OverallDuelLab GameBench 2–73.0†
GameBench 2 by reasoning settingDuelLab GameBench 2–79.4†
Design Arena WebsiteDesign Arena–1290†
FrontierCode 2.0 (AA)Artificial Analysis–35.6%
Agents & tools
Terminal-BenchTerminal-Bench–50.3%
APEX-AgentsMercor–54.3%
Arena AgentArena (LMArena)–6.9
LiveBench Agentic CodingLiveBench–52.9%
Terminal-Bench 2.1 (Vals)Vals AI–83.2%
GDPval-AA v2.1, normalised scoreArtificial Analysis41.5%–
GDPval-AA v2.1Artificial Analysis13351514
τ³-Banking (AA)Artificial Analysis35.7%–
ITBench SRE (AA)Artificial Analysis–49.4%
CyberBenchVals AI–78.0%
Finance Agent v2Vals AI–49.0%
Harvey's Legal Agent Benchmark v1Vals AI–1.7%
Legal Research BenchVals AI–28.9%
SREBenchVals AI–30.5%
Tax Agent BenchVals AI–15.0%
Terminal-Bench 4.0 (Vals)Vals AI–44.4%
Terminal-Bench ScienceVals AI–30.0%
Arena SearchArena (LMArena)–1229
Arena Agent Task CompletionArena (LMArena)–4.5
Vending-Bench 2Andon Labs–14427.9†
SpireClimbEpoch AI Benchmarking Hub–9.2%
AutomationBenchArtificial Analysis–61.6%
GDP.pdf (AA)Artificial Analysis–25.2%
Harvey LAB-AA v1.1Harvey–3.6%
AA-Briefcase v1.1Artificial Analysis–1476
Design Arena AgenticDesign Arena–1218†
Terminal-Bench-Science 0.1 (AA)Artificial Analysis–30.0%
CWE-Bench (AA)Artificial Analysis–64.2%
DeepSec-Bench (AA)Artificial Analysis–46.3%
CyberGym end-to-end (AA)Artificial Analysis–0.0%
FACTS SearchGoogle–92.7%†
Maths
FrontierMath Tiers 1–3Epoch AI Benchmarking Hub–89.8%
FrontierMath Tier 4Epoch AI Benchmarking Hub–90.0%
OTIS Mock AIMEEpoch AI Benchmarking Hub–100.0%
ProofBenchVals AI–83.0%†
LiveBench MathematicsLiveBench–96.4%
Arena MathsArena (LMArena)–1440
FrontierMath ErdősEpoch AI Benchmarking Hub–1.5%
Knowledge
SimpleQA VerifiedEpoch AI Benchmarking Hub–60.7%
LiveBench 1.0 Data AnalysisLiveBench–81.2%
AA-Omniscience v1.1Artificial Analysis–21.4
PRBench LegalScale AI SEAL–38.6%†
BioMysteryBenchVals AI–74.8%
Excel Modeling BenchmarkVals AI–71.5%
MedCodeVals AI–47.1%
MedScribeVals AI–82.0%
Public Benefits Bench v1.1Vals AI–56.6%
Financial Coaching BenchVals AI–62.6%
EBR-benchEpoch AI Benchmarking Hub–53.3%
GDP.pdfEpoch AI Benchmarking Hub–26.4%
AA-Omniscience v1.1: accuracyArtificial Analysis–51.1%
AA-Omniscience v1.1: non-hallucinationArtificial Analysis–39.1%
HealthBench Professional (AA)OpenAI–62.3%
FACTS ParametricGoogle–76.1%†
Instruction following
LiveBench LanguageLiveBench–85.3%
LiveBench Instruction FollowingLiveBench–68.6%
Arena Instruction FollowingArena (LMArena)–1449
Human preference
Arena TextArena (LMArena)–1457
EQ-Bench Creative Writing v3EQ-Bench–2126†
Arena Creative WritingArena (LMArena)–1428
Arena Multi-turnArena (LMArena)–1468
Arena Longer QueriesArena (LMArena)–1466
Multimodal
Arena VisionArena (LMArena)–1233
MMMU-ProArtificial Analysis75.4%83.0%
Humanity's Sixth SenseScale AI SEAL–31.2%
Blueprint-Bench 2Epoch AI Benchmarking Hub–36.9%†
Furniture AssemblyEpoch AI Benchmarking Hub–58.3%
AA-3DArtificial Analysis–1131
FACTS MultimodalGoogle–43.9%†
Long context
AA-LCR v1.1Artificial Analysis83.0%83.7%
MLCRArtificial Analysis–33.9%
FACTS GroundingGoogle–83.3%†
Composite
Epoch Capabilities IndexEpoch AI Benchmarking Hub–163.8†
LiveBench 1.0LiveBench–79.3%
AA Intelligence Index v5.0.1Artificial Analysis–44.2
AA Intelligence Index v4.3.2Artificial Analysis35.247.6
AA Progress Index v1Artificial Analysis299.1334
Vals RSI IndexVals AI–28.1%
Vals Science IndexVals AI–45.0%
AA Cyber IndexArtificial Analysis–36.8
Safety & honesty
FORTRESSScale AI SEAL–13.5%†
DistressBenchScale AI SEAL–68.0%†

Data as of 2026-11-16. Best configuration of each model; every score links to its source on the model pages.

DuelLab GameBench 2, release gb2-20261111-7b598bb13de4 (https://benchmarks.duellab.org/releases/gb2-20261111-7b598bb13de4/), CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Scores mapped to BenchLeader model names; not endorsed by DuelLab.

What changed

  • GPT-6 Sol (max): 1 new result (GDPval-AA v2.1).
  • GPT-6 Sol (max): 1 new result (MirrorCode).
  • GPT-6 Sol (max): BenchLeader Index 63.5 → 62.9.
  • GPT-6 Sol (max): 2 score revisions.
  • GPT-6 Sol (max): 1 score revision.
  • GPT-6 Sol (max): 1 score revision.
  • GPT-6 Sol (max): 6 new results (AA Intelligence Index v5.0.1, AA-Omniscience v1.1, CritPt v2 hard subset (AA)…).
  • GPT-6 Sol (max): 1 score revision.
  • GPT-6 Sol (max): 1 score revision.
  • GPT-6 Sol (max): BenchLeader Index 64.1 → 63.5.

Rankings: Best LLM overall · Best LLM for agents · Best cheap LLMs

Agnes 2.5 Pro Beta vs GPT-6 Sol: questions

Agnes 2.5 Pro Beta vs GPT-6 Sol: which is cheaper, and will that change?
Agnes 2.5 Pro Beta is cheaper today: $0.150 against $4.00 per million tokens, blended at three input tokens per output token. No price change has been announced for either.
Is Agnes 2.5 Pro Beta or GPT-6 Sol better for agents and tool use?
Agnes 2.5 Pro Beta scores higher in agentic tasks (64 vs 59 on the category index, where 50 is average).
Which is better for long documents?
They are level on long-context benchmarks (66 vs 67). Both take about 1.1M tokens: Agnes 2.5 Pro Beta 1,000,000, GPT-6 Sol 1,050,000.
Do independent benchmarks agree on Agnes 2.5 Pro Beta vs GPT-6 Sol?
Across the 7 benchmarks both have, Agnes 2.5 Pro Beta leads 0, GPT-6 Sol leads 7.
When was GPT-6 Sol released?
GPT-6 Sol was released on 22 Sep 2026.