BenchLeader
Menu
Best LLMsChoose a modelLeaderboardsModelsBenchmarksProvidersReleases9

SWE-bench Verified (bash only)

Official SWE-bench Verified results from the minimal bash-only mini-SWE-agent harness.

As of 25 Oct 2026, Claude Opus 4.5 leads SWE-bench Verified (bash only) on BenchLeader with 76.8%, ahead of Gemini 3 Flash at 75.8%, across 45 model configurations with a published result.

Published by
SWE-bench
Category
Coding
Index weight
1.0
Models
45
Data as of
25 Oct 2026
Newest result
26 Feb 2026

The publisher has posted no new results on this board since 26 Feb 2026. BenchLeader re-reads it every day, so newer models appear here as soon as they are published.

SWE-bench (Jimenez et al. 2024), swebench.com.

What the test looks like

SWE-bench Verified is 500 real GitHub issues from popular Python repositories, each screened by human annotators to make sure the task is well specified and the tests are fair. The model reads the issue, edits the repository, and its patch is accepted only if the hidden tests pass.

How it is scored

Percent of issues resolved. This board uses the minimal, bash-only mini-SWE-agent scaffold that the SWE-bench maintainers run themselves, so every model gets the same simple agent loop.

What to keep in mind

Results depend heavily on the agent scaffold: the same model can score fifteen points higher in a richer harness, which is why the any-scaffold board is shown separately. All tasks are Python, and the repositories are well known, so some memorisation is possible.

Top 15
  1. 1Claude Opus 4.5 (high)76.8%
  2. 2Gemini 3 Flash (high)75.8%
  3. 3MiniMax M2.5 (high)75.8%
  4. 4Claude Opus 4.675.6%
  5. 5Claude Opus 4.5 (medium)74.4%
  6. 6Gemini 3 Pro74.2%
  7. 7GPT-5.2-Codex72.8%
  8. 8GPT-5.2 (high)72.8%
  9. 9GLM-5 (high)72.8%
  10. 10Claude Sonnet 4.5 (high)71.4%
  11. 11Kimi K2.5 (high)70.8%
  12. 12Claude Sonnet 4.570.6%
  13. 13Gemini 3 Pro (high)69.6%
  14. 14GPT-5.269.0%
  15. 15Claude Opus 467.6%
45 of 45
#
1Claude Opus 4.5high76.8%59.2mini-SWE-agent$376.952026-02-17
2Gemini 3 Flashhigh75.8%58.3mini-SWE-agent$177.982026-02-17
3MiniMax M2.5high75.8%–mini-SWE-agent$36.642026-02-17
4Claude Opus 4.675.6%62.5mini-SWE-agent$275.762026-02-17
5Claude Opus 4.5medium74.4%–mini-SWE-agent$360.622025-11-24
6Gemini 3 Pro74.2%60.4mini-SWE-agent$229.982025-11-18
7GPT-5.2-Codex72.8%57.3mini-SWE-agent$224.712026-02-19
8GPT-5.2high72.8%56.0mini-SWE-agent$236.782026-02-17
9GLM-5high72.8%–mini-SWE-agent$267.192026-02-17
10Claude Sonnet 4.5high71.4%58.1mini-SWE-agent$328.952026-02-17
11Kimi K2.5high70.8%–mini-SWE-agent$73.282026-02-17
12Claude Sonnet 4.570.6%53.7mini-SWE-agent$279.172025-09-29
13Gemini 3 Prohigh69.6%60.0mini-SWE-agent$480.012026-02-26
14GPT-5.269.0%58.0mini-SWE-agent$134.832025-12-11
15Claude Opus 467.6%52.6mini-SWE-agent$565.642025-08-02
16Claude Haiku 4.5high66.6%–mini-SWE-agent$165.462026-02-17
17GPT-5.1medium66.0%52.3mini-SWE-agent$153.082025-11-20
18GPT-5.1-Codexmedium66.0%–mini-SWE-agent$294.442025-11-24
19GPT-5medium65.0%57.5mini-SWE-agent$140.192025-08-07
20Claude Sonnet 464.9%50.5mini-SWE-agent$185.732025-07-26
21Kimi K2 Thinkingthinking63.4%52.4mini-SWE-agent$219.152025-12-10
22MiniMax-M261.0%50.8mini-SWE-agent$214.132025-11-24
23DeepSeek V3.2 Reasoner60.0%–mini-SWE-agent$14.042025-12-01
24GPT-5 minimedium59.8%53.8mini-SWE-agent$17.742025-08-07
25o358.4%59.5mini-SWE-agent$166.832025-07-26
26Devstral Small 256.4%39.5mini-SWE-agent–2025-12-09
27GPT-5 mini56.2%56.5mini-SWE-agent$23.602026-02-17
28GLM-4.655.4%49.9mini-SWE-agent$48.312025-12-01
29Qwen3 Coder 480B55.4%48.0mini-SWE-agent$123.962025-08-02
30GLM-4.554.2%51.7mini-SWE-agent$148.602025-08-22
31Devstral 253.8%42.3mini-SWE-agent–2025-12-09
32Gemini 2.5 Pro53.6%53.9mini-SWE-agent$144.192025-07-26
33Claude 3.7 Sonnet52.8%50.0mini-SWE-agent$177.312025-07-20
34o4-mini45.0%56.9mini-SWE-agent$104.992025-07-26
35Kimi K243.8%50.4mini-SWE-agent$265.802025-08-07
36GPT-4.139.6%48.3mini-SWE-agent$73.162025-07-26
37GPT-5 nanomedium34.8%45.9mini-SWE-agent$19.042025-08-07
38Gemini 2.5 Flash28.7%52.7mini-SWE-agent$66.942025-07-26
39gpt-oss-120b26.0%45.6mini-SWE-agent$28.562025-08-07
40GPT-4.1 mini23.9%45.0mini-SWE-agent$217.582025-07-20
41GPT-4o21.6%43.9mini-SWE-agent$765.572025-07-20
42Llama 4 Maverick21.0%42.8mini-SWE-agent$156.852025-07-20
43Gemini 2.0 Flash13.5%43.2mini-SWE-agent–2025-07-26
44Llama 4 Scout9.1%39.7mini-SWE-agent$57.822025-07-20
45Qwen2.5 Coder 32B Instruct9.0%42.1mini-SWE-agent$34.062025-08-03

More coding benchmarks · All LLM benchmarks

Cite as: BenchLeader, “SWE-bench Verified (bash only) leaderboard”, data as of 25 Oct 2026, https://www.benchleader.com/benchmarks/swe_bench_verified.

SWE-bench Verified (bash only): questions

What does SWE-bench Verified (bash only) measure?
SWE-bench Verified is 500 real GitHub issues from popular Python repositories, each screened by human annotators to make sure the task is well specified and the tests are fair. The model reads the issue, edits the repository, and its patch is accepted only if the hidden tests pass. Scores are reported in percent of tasks solved; higher is better.
Which AI model leads SWE-bench Verified (bash only)?
Claude Opus 4.5 leads SWE-bench Verified (bash only) with 76.8% as of 25 Oct 2026, ahead of Gemini 3 Flash at 75.8%.
How many models have SWE-bench Verified (bash only) results?
45 model configurations have a SWE-bench Verified (bash only) result on BenchLeader, all taken from SWE-bench.
Who runs SWE-bench Verified (bash only) and how often is it updated?
SWE-bench Verified (bash only) is published by SWE-bench. BenchLeader re-reads the published results once a day and records the date each result was published.
Does SWE-bench Verified (bash only) count toward the BenchLeader Index?
Yes. SWE-bench Verified (bash only) contributes to the coding category of the BenchLeader Index, normalised so that 50 is the average of the evaluated models.