Kagi LLM Benchmark
Kagi's unpublished reasoning, coding and instruction questions.
As of 24 Oct 2026, Claude Fable 5 leads Kagi LLM Benchmark on BenchLeader with 91.4%, ahead of Claude Fable 5 at 88.8%, across 136 model configurations with a published result.
- Published by
- Kagi LLM Benchmark
- Category
- Reasoning
- Index weight
- 0.5
- Models
- 136
- Data as of
- 24 Oct 2026
Kagi (kagi.com).
What the test looks like
Kagi's benchmark uses a private, changing set of reasoning, coding and instruction questions written to be unusual, so that memorised answers do not help.
How it is scored
Percent correct, run by Kagi with its own grading.
What to keep in mind
Private questions cannot be inspected, the set is small, and it changes over time; treat it as one more independent opinion.
- 1Claude Fable 5 (thinking)91.4%
- 2Claude Fable 588.8%
- 3GPT-5.588.8%
- 4Claude Opus 4.8 (thinking)88.8%
- 5Claude Opus 4.6 (thinking)83.6%
- 6Grok 4.583.5%
- 7Claude Opus 4.7 (thinking)80.7%
- 8Claude Opus 4.5 (thinking)80.2%
- 9Gemini 3 Pro80.1%
- 10Kimi K2.5 (thinking)78.5%
- 11GPT-5 Pro76.8%
- 12GLM-5 (thinking)75.0%
- 13Grok 4.20 (thinking)75.0%
- 14GPT-4o75.0%
- 15Claude Opus 4 (thinking)74.3%
136 of 136
| # | ||||
|---|---|---|---|---|
| 1 |  Claude Fable 5thinking | 91.4% | – | $10.40 |
| 2 | Claude Fable 5 | 88.8% | 67.1 | $9.90 |
| 3 |  GPT-5.5 | 88.8% | 60.3 | $2.20 |
| 4 | Claude Opus 4.8thinking | 88.8% | – | $8.80 |
| 5 | Claude Opus 4.6thinking | 83.6% | 62.5 | $4.80 |
| 6 |  Grok 4.5 | 83.5% | 59.0 | $1.00 |
| 7 | Claude Opus 4.7thinking | 80.7% | – | $2.80 |
| 8 | Claude Opus 4.5thinking | 80.2% | 59.9 | $4.50 |
| 9 |  Gemini 3 Pro | 80.1% | 60.4 | $0.400 |
| 10 |  Kimi K2.5thinkingopen ↗ | 78.5% | 57.6 | $0.500 |
| 11 | GPT-5 Pro | 76.8% | 59.8 | $31.80 |
| 12 |  GLM-5thinkingopen ↗ | 75.0% | 58.9 | $2.00 |
| 13 | Grok 4.20thinking | 75.0% | 58.6 | $1.30 |
| 14 | GPT-4o | 75.0% | 43.9 | $0.800 |
| 15 | Claude Opus 4thinking | 74.3% | 55.0 | $22.40 |
| 16 |  Qwen3.5 397B A17Bopen ↗ | 73.7% | 51.5 | $1.80 |
| 17 | Grok 4 | 73.6% | 58.0 | $1.00 |
| 18 | Claude Opus 4.7 | 73.3% | 63.3 | $2.20 |
| 19 | GPT-5.2 | 73.3% | 58.0 | $1.40 |
| 20 | Claude Sonnet 4thinking | 73.0% | 53.2 | $5.40 |
| 21 | GPT-5 | 72.7% | 56.2 | $7.10 |
| 22 | Qwen3 Maxmax | 72.5% | 52.8 | $1.90 |
| 23 | Claude Opus 4.6 | 72.4% | 62.6 | $1.60 |
| 24 | o3-pro | 72.1% | 53.4 | $34.20 |
| 25 | Claude Opus 4.5 | 70.7% | 57.9 | $1.40 |
| 26 | GPT-5 mini | 70.3% | 56.5 | $4.90 |
| 27 | Gemini 2.5 Pro | 70.3% | 53.9 | $1.70 |
| 28 | GPT-5-Codex | 70.3% | – | $4.60 |
| 29 |  DeepSeek R1open ↗ | 69.4% | 46.8 | $9.90 |
| 30 | Qwen3 235B A22Bthinkingopen ↗ | 69.4% | 42.3 | $0.100 |
| 31 | Claude Haiku 4thinking | 69.0% | – | $0.800 |
| 32 | o3 | 67.6% | 59.5 | $4.80 |
| 33 | o4-mini | 67.6% | 56.9 | $3.10 |
| 34 | Gemini 3.1 Flash Lite | 67.2% | 53.7 | – |
| 35 | GPT-5.6 Sol | 67.0% | 55.9 | $2.70 |
| 36 | Qwen3-Next 80B A3Bthinkingopen ↗ | 66.7% | 49.3 | $1.00 |
| 37 |  Ring 1topen | 66.4% | 46.1 | $1.00 |
| 38 | Grok 4 Fastthinking | 66.1% | 51.4 | $0.300 |
| 39 | Kimi K2 Thinkingthinkingopen ↗ | 64.4% | 52.4 | $0.800 |
| 40 | GPT-5.4 | 63.8% | 58.2 | $0.300 |
| 41 | Kimi K2.5open ↗ | 63.8% | 54.5 | $0.100 |
| 42 | Gemma 4 31Bopen ↗ | 63.5% | 48.4 | $1.00 |
| 43 | Qwen-Plus | 63.3% | 48.5 | $1.10 |
| 44 | GLM 5.2open ↗ | 62.6% | 52.0 | $0.700 |
| 45 |  Step 3 | 62.3% | 51.5 | $1.60 |
| 46 | GPT-5 nano | 62.2% | 55.9 | $0.400 |
| 47 | Grok 3 mini | 61.3% | 52.3 | $0.300 |
| 48 | Grok 3 | 61.3% | 51.5 | $2.60 |
| 49 |  Cogito v2 Llama 405B | 61.1% | – | $2.10 |
| 50 | GLM 5.2thinkingopen ↗ | 60.0% | – | $1.00 |
| 51 | GLM 4.5Vopen ↗ | 59.8% | 50.9 | $0.700 |
| 52 | Claude Opus 4 | 59.6% | 52.6 | $8.40 |
| 53 | gpt-oss-120bopen ↗ | 58.6% | 45.6 | $0.400 |
| 54 | Cogito v2 Deepseek 671B | 58.4% | – | $0.700 |
| 55 | Claude Sonnet 4.5 | 57.9% | 53.7 | $1.00 |
| 56 | GLM-4.5open ↗ | 57.9% | 51.7 | $1.00 |
| 57 |  MiniMax-M2open ↗ | 57.8% | 50.8 | $0.100 |
| 58 | Ling 1topen | 57.8% | 43.7 | $0.200 |
| 59 | DeepSeek V3.1 Terminusopen ↗ | 57.4% | 53.2 | $0.200 |
| 60 | Gemini 2.5 Flashthinking | 56.8% | 47.7 | $0.500 |
More reasoning benchmarks · All LLM benchmarks
Cite as: BenchLeader, “Kagi LLM Benchmark leaderboard”, data as of 24 Oct 2026, https://www.benchleader.com/benchmarks/kagi_llm.
Kagi LLM Benchmark: questions
- What does Kagi LLM Benchmark measure?
- Kagi's benchmark uses a private, changing set of reasoning, coding and instruction questions written to be unusual, so that memorised answers do not help. Scores are reported in percent of tasks solved; higher is better.
- Which AI model leads Kagi LLM Benchmark?
- Claude Fable 5 leads Kagi LLM Benchmark with 91.4% as of 24 Oct 2026, ahead of Claude Fable 5 at 88.8%.
- How many models have Kagi LLM Benchmark results?
- 136 model configurations have a Kagi LLM Benchmark result on BenchLeader, all taken from Kagi LLM Benchmark.
- Who runs Kagi LLM Benchmark and how often is it updated?
- Kagi LLM Benchmark is published by Kagi LLM Benchmark. BenchLeader re-reads the published results once a day and records the date each result was published.
- Does Kagi LLM Benchmark count toward the BenchLeader Index?
- Yes. Kagi LLM Benchmark contributes to the reasoning category of the BenchLeader Index, normalised so that 50 is the average of the evaluated models.