Aider Polyglot
Percent of 225 Exercism problems solved on the second attempt.
As of 25 Oct 2026, GPT-5 leads Aider Polyglot on BenchLeader with 88.0%, ahead of o3-pro at 84.9%, across 48 model configurations with a published result.
- Published by
- Aider polyglot leaderboard
- Category
- Coding
- Index weight
- 1.0
- Models
- 48
- Data as of
- 25 Oct 2026
- Newest result
- 3 Oct 2025
The publisher has posted no new results on this board since 3 Oct 2025. BenchLeader re-reads it every day, so newer models appear here as soon as they are published.
Apache 2.0 — Aider-AI/aider.
What the test looks like
Aider's polyglot benchmark asks the model to solve 225 of the hardest Exercism exercises across C++, Go, Java, JavaScript, Python and Rust, editing files through the Aider tool and running the tests.
How it is scored
Percent of exercises passing tests within two attempts; the model sees test output after a first failure.
What to keep in mind
It tests code editing in a specific tool, and Aider's own edit formats favour some models. Exercism exercises are public.
- 1GPT-588.0%
- 2o3-pro84.9%
- 3Gemini 2.5 Pro83.1%
- 4o381.3%
- 5Grok 479.6%
- 6DeepSeek V3.274.2%
- 7o4-mini72.0%
- 8Claude Opus 472.0%
- 9DeepSeek R1 052871.4%
- 10Claude 3.7 Sonnet64.9%
- 11Deepseek R1 Plus Claude 3.5 Sonnet64.0%
- 12o161.7%
- 13Claude Sonnet 461.3%
- 14o3-mini60.4%
- 15Claude 3.7 Sonnet (none)60.4%
| # | |||||
|---|---|---|---|---|---|
| 1 |  GPT-5 | 88.0% | 56.2 | $29.08 | 2025-08-23 |
| 2 | o3-pro | 84.9% | 53.4 | $146.32 | 2025-06-28 |
| 3 |  Gemini 2.5 Pro | 83.1% | 53.9 | $49.88 | 2025-06-06 |
| 4 | o3 | 81.3% | 59.5 | $21.23 | 2025-06-25 |
| 5 |  Grok 4 | 79.6% | 58.0 | $59.62 | 2025-07-11 |
| 6 |  DeepSeek V3.2open ↗ | 74.2% | 56.6 | $1.30 | 2025-10-03 |
| 7 | o4-mini | 72.0% | 56.9 | $19.64 | 2025-04-16 |
| 8 |  Claude Opus 4 | 72.0% | 52.6 | $65.75 | 2025-05-25 |
| 9 | DeepSeek R1 0528open ↗ | 71.4% | 49.5 | $4.80 | 2025-06-06 |
| 10 | Claude 3.7 Sonnet | 64.9% | 50.0 | $36.83 | 2025-02-24 |
| 11 | Deepseek R1 Plus Claude 3.5 Sonnet | 64.0% | – | $13.29 | 2025-01-23 |
| 12 | o1 | 61.7% | 53.1 | $186.50 | 2024-12-21 |
| 13 | Claude Sonnet 4 | 61.3% | 50.5 | $26.58 | 2025-05-24 |
| 14 | o3-mini | 60.4% | 49.9 | $18.16 | 2025-01-31 |
| 15 | Claude 3.7 Sonnetno reasoning | 60.4% | 49.5 | $17.72 | 2025-02-24 |
| 16 |  Qwen3 235B A22Bopen ↗ | 59.6% | 50.1 | – | 2025-05-09 |
| 17 |  Kimi K2open ↗ | 59.1% | 50.4 | $1.24 | 2025-07-17 |
| 18 | DeepSeek R1open ↗ | 56.9% | 46.8 | $5.42 | 2025-01-20 |
| 19 | DeepSeek V3 0324open ↗ | 55.1% | 53.5 | $1.12 | 2025-03-24 |
| 20 | Gemini 2.5 Flash | 55.1% | 52.7 | $8.56 | 2025-05-25 |
| 21 | Quasar Alpha | 54.7% | – | – | 2025-04-04 |
| 22 | Grok 3 | 53.3% | 51.5 | $11.03 | 2025-04-10 |
| 23 | Optimus Alpha | 52.9% | – | – | 2025-04-10 |
| 24 | GPT-4.1 | 52.4% | 48.3 | $9.86 | 2025-04-14 |
| 25 | Claude 3.5 Sonnet | 51.6% | 46.3 | $14.41 | 2025-01-17 |
| 26 | Grok 3 mini | 49.3% | 52.3 | $0.735 | 2025-04-10 |
| 27 | DeepSeek V3open ↗ | 48.4% | 44.0 | $0.337 | 2024-12-25 |
| 28 | GPT-4o | 45.3% | 43.9 | $19.74 | 2025-03-29 |
| 29 | GPT-4.5 | 44.9% | 49.7 | $183.18 | 2025-02-27 |
| 30 | gpt-oss-120bopen ↗ | 41.8% | 45.6 | $0.741 | 2025-08-06 |
| 31 | Qwen3 32Bopen ↗ | 40.0% | 51.5 | $0.760 | 2025-05-08 |
| 32 | Gemini 1206 | 38.2% | – | – | 2024-12-22 |
| 33 | Gemini 2.0 Pro | 35.6% | 47.0 | – | 2025-02-25 |
| 34 | o1-mini | 32.9% | 46.1 | $18.58 | 2024-12-22 |
| 35 | GPT-4.1 mini | 32.4% | 45.0 | $1.99 | 2025-04-14 |
| 36 | Claude 3.5 Haiku | 28.0% | 39.7 | $6.06 | 2024-12-21 |
| 37 | Qwq 32B Plus Qwen 2.5 Coder | 26.2% | – | – | 2025-03-07 |
| 38 | Gemini 2.0 Flash | 22.2% | 43.2 | – | 2024-12-22 |
| 39 | QwQ-32Bopen ↗ | 20.9% | 45.4 | – | 2025-03-06 |
| 40 | Gemini 2.0 Flash Thinkingthinking | 18.2% | 44.9 | – | 2025-01-21 |
| 41 | Qwen2.5 Coder 32B Instructopen ↗ | 16.4% | 42.1 | – | 2024-12-26 |
| 42 |  Llama 4 Maverickopen ↗ | 15.6% | 42.8 | – | 2025-04-06 |
| 43 |  Yi Lightning | 12.9% | 46.8 | – | 2024-12-23 |
| 44 |  Command A 03 2025 Quality | 12.0% | – | – | 2025-03-14 |
| 45 |  Openhands Lm 32B | 10.2% | – | – | 2025-04-19 |
| 46 | GPT-4.1 nano | 8.9% | 38.0 | $0.428 | 2025-04-14 |
| 47 | Gemma 3 27Bopen ↗ | 4.9% | 38.9 | – | 2025-03-15 |
| 48 | GPT-4o mini | 3.6% | 35.8 | $0.324 | 2024-12-21 |
More coding benchmarks · All LLM benchmarks
Cite as: BenchLeader, “Aider Polyglot leaderboard”, data as of 25 Oct 2026, https://www.benchleader.com/benchmarks/aider_polyglot.
Aider Polyglot: questions
- What does Aider Polyglot measure?
- Aider's polyglot benchmark asks the model to solve 225 of the hardest Exercism exercises across C++, Go, Java, JavaScript, Python and Rust, editing files through the Aider tool and running the tests. Scores are reported in percent of tasks solved; higher is better.
- Which AI model leads Aider Polyglot?
- GPT-5 leads Aider Polyglot with 88.0% as of 25 Oct 2026, ahead of o3-pro at 84.9%.
- How many models have Aider Polyglot results?
- 48 model configurations have a Aider Polyglot result on BenchLeader, all taken from Aider polyglot leaderboard.
- Who runs Aider Polyglot and how often is it updated?
- Aider Polyglot is published by Aider polyglot leaderboard. BenchLeader re-reads the published results once a day and records the date each result was published.
- Does Aider Polyglot count toward the BenchLeader Index?
- Yes. Aider Polyglot contributes to the coding category of the BenchLeader Index, normalised so that 50 is the average of the evaluated models.