SWE-bench Verified (bash only)
Official SWE-bench Verified results from the minimal bash-only mini-SWE-agent harness.
- Published by
- SWE-bench
- Category
- Coding
- Index weight
- 1.0
- Models
- 44
- Data as of
- 9 Sept 2026
SWE-bench (Jimenez et al. 2024), swebench.com.
- 1Claude Opus 4.576.8%
- 2Gemini 3 Flash75.8%
- 3MiniMax-M2.575.8%
- 4Claude Opus 4.675.6%
- 5Claude Opus 4.574.4%
- 6Gemini 3 Pro74.2%
- 7GPT-5.2 Codex72.8%
- 8GPT-5.272.8%
- 9GLM-572.8%
- 10Claude Sonnet 4.571.4%
- 11Kimi K2.570.8%
- 12Claude Sonnet 4.570.6%
- 13Gemini 3 Pro69.6%
- 14GPT-5.269.0%
- 15Claude Opus 467.6%
44 of 44
| # | |||||
|---|---|---|---|---|---|
| 1 |  Claude Opus 4.5Anthropic | 76.8% | – | 56.6 | – |
| 2 |  Gemini 3 FlashGoogle | 75.8% | – | 58.5 | – |
| 3 |  MiniMax-M2.5MiniMaxopen | 75.8% | – | – | – |
| 4 | Claude Opus 4.6Anthropic | 75.6% | – | 63.1 | – |
| 5 | Claude Opus 4.5Anthropic | 74.4% | – | – | – |
| 6 | Gemini 3 ProGoogle | 74.2% | – | 61.6 | – |
| 7 |  GPT-5.2 CodexOpenAI | 72.8% | – | 60.7 | – |
| 8 | GPT-5.2OpenAI | 72.8% | – | 55.5 | – |
| 9 |  GLM-5Zhipu AIopen | 72.8% | – | – | – |
| 10 | Claude Sonnet 4.5Anthropic | 71.4% | – | 57.8 | – |
| 11 |  Kimi K2.5Moonshot AIopen | 70.8% | – | – | – |
| 12 | Claude Sonnet 4.5Anthropic | 70.6% | – | 52.3 | – |
| 13 | Gemini 3 ProGoogle | 69.6% | – | 57.0 | – |
| 14 | GPT-5.2OpenAI | 69.0% | – | 61.5 | – |
| 15 | Claude Opus 4Anthropic | 67.6% | – | 50.7 | – |
| 16 | Claude Haiku 4.5Anthropic | 66.6% | – | – | – |
| 17 | GPT-5.1OpenAI | 66.0% | – | 52.8 | – |
| 18 | GPT-5.1-CodexOpenAI | 66.0% | – | – | – |
| 19 | GPT-5OpenAI | 65.0% | – | 58.5 | – |
| 20 | Claude Sonnet 4Anthropic | 64.9% | – | 49.5 | – |
| 21 | Kimi K2 ThinkingMoonshot AIopen | 63.4% | – | 52.8 | – |
| 22 | MiniMax-M2MiniMaxopen | 61.0% | – | 55.1 | – |
| 23 |  DeepSeek V3.2 Reasonerdeepseekopen | 60.0% | – | – | – |
| 24 | GPT-5 miniOpenAI | 59.8% | – | 54.5 | – |
| 25 | o3OpenAI | 58.4% | – | 60.3 | – |
| 26 |  Devstral SmallMistral AIopen | 56.4% | – | 42.5 | – |
| 27 | GPT-5 miniOpenAI | 56.2% | – | 55.8 | – |
| 28 |  Qwen3 Coder 480BAlibabaopen | 55.4% | – | 48.5 | – |
| 29 | GLM-4.6Zhipu AIopen | 55.4% | – | 47.6 | – |
| 30 | GLM-4.5Zhipu AIopen | 54.2% | – | 50.2 | – |
| 31 | Gemini 2.5 ProGoogle | 53.6% | – | 54.1 | – |
| 32 | Claude 3.7 SonnetAnthropic | 52.8% | – | 49.8 | – |
| 33 | o4-miniOpenAI | 45.0% | – | 54.9 | – |
| 34 | Kimi K2Moonshot AIopen | 43.8% | – | 48.8 | – |
| 35 | GPT-4.1OpenAI | 39.6% | – | 47.3 | – |
| 36 | GPT-5 nanoOpenAI | 34.8% | – | 46.5 | – |
| 37 | Gemini 2.5 FlashGoogle | 28.7% | – | 48.8 | – |
| 38 | gpt-oss-120bOpenAIopen | 26.0% | – | 47.6 | – |
| 39 | GPT-4.1 miniOpenAI | 23.9% | – | 44.5 | – |
| 40 | GPT-4oOpenAI | 21.6% | – | 42.3 | – |
| 41 |  Llama 4 MaverickMetaopen | 21.0% | – | 42.6 | – |
| 42 | Gemini 2.0 FlashGoogle | 13.5% | – | 42.3 | – |
| 43 | Llama 4 ScoutMetaopen | 9.1% | – | 39.7 | – |
| 44 | Qwen2.5-Coder 32B InstructAlibabaopen | 9.0% | – | 42.2 | – |