BenchLeader

SWE-bench Verified (bash only) leaderboard

As of 7 Nov 2026, the top score on SWE-bench Verified (bash only) is 76.8% by Claude Opus 4.5 (Anthropic), ahead of Gemini 3 Flash at 75.8%, across 45 model configurations with a published result (SWE-bench, result dated 17 Feb 2026). SWE-bench Verified is 500 real GitHub issues from popular Python repositories, each screened by human annotators to make sure the task is well specified and the tests are fair. The model reads the issue, edits the repository, and its patch is accepted only if the hidden tests pass. SWE-bench Verified (bash only) counts toward the BenchLeader coding score.

Key facts (as of 7 Nov 2026)

  • Claude Opus 4.5 holds the top SWE-bench Verified (bash only) score at 76.8% (SWE-bench, 17 Feb 2026).
  • Gemini 3 Flash is second on SWE-bench Verified (bash only) at 75.8% (SWE-bench, 17 Feb 2026).
  • SWE-bench Verified (bash only): Official SWE-bench Verified results from the minimal bash-only mini-SWE-agent harness (SWE-bench).
  • 45 model configurations have a published SWE-bench Verified (bash only) result on BenchLeader (BenchLeader, 7 Nov 2026).
  • SWE-bench Verified (bash only) feeds the coding category of the BenchLeader Index (BenchLeader methodology).

Official SWE-bench Verified results from the minimal bash-only mini-SWE-agent harness.

Published by
SWE-bench
Category
Coding
Index weight
1.0
Models
45
Data as of
7 Nov 2026

SWE-bench (Jimenez et al. 2024), swebench.com.

What the test looks like

SWE-bench Verified is 500 real GitHub issues from popular Python repositories, each screened by human annotators to make sure the task is well specified and the tests are fair. The model reads the issue, edits the repository, and its patch is accepted only if the hidden tests pass.

How it is scored

Percent of issues resolved. This board uses the minimal, bash-only mini-SWE-agent scaffold that the SWE-bench maintainers run themselves, so every model gets the same simple agent loop.

What to keep in mind

Results depend heavily on the agent scaffold: the same model can score fifteen points higher in a richer harness, which is why the any-scaffold board is shown separately. All tasks are Python, and the repositories are well known, so some memorisation is possible.

45 of 45
#
1Claude Opus 4.5high76.8%59.0mini-SWE-agent$376.952026-02-17
2Gemini 3 Flashhigh75.8%58.3mini-SWE-agent$177.982026-02-17
3MiniMax M2.5high75.8%–mini-SWE-agent$36.642026-02-17
4Claude Opus 4.675.6%62.3mini-SWE-agent$275.762026-02-17
5Claude Opus 4.5medium74.4%–mini-SWE-agent$360.622025-11-24
6Gemini 3 Pro74.2%60.2mini-SWE-agent$229.982025-11-18
7GPT-5.2-Codex72.8%57.3mini-SWE-agent$224.712026-02-19
8GPT-5.2high72.8%55.9mini-SWE-agent$236.782026-02-17
9GLM-5high72.8%–mini-SWE-agent$267.192026-02-17
10Claude Sonnet 4.5high71.4%57.9mini-SWE-agent$328.952026-02-17
11Kimi K2.5high70.8%–mini-SWE-agent$73.282026-02-17
12Claude Sonnet 4.570.6%53.7mini-SWE-agent$279.172025-09-29
13Gemini 3 Prohigh69.6%60.1mini-SWE-agent$480.012026-02-26
14GPT-5.269.0%57.9mini-SWE-agent$134.832025-12-11
15Claude Opus 467.6%52.6mini-SWE-agent$565.642025-08-02
16Claude Haiku 4.5high66.6%–mini-SWE-agent$165.462026-02-17
17GPT-5.1medium66.0%52.2mini-SWE-agent$153.082025-11-20
18GPT-5.1-Codexmedium66.0%–mini-SWE-agent$294.442025-11-24
19GPT-5medium65.0%58.1mini-SWE-agent$140.192025-08-07
20Claude Sonnet 464.9%50.5mini-SWE-agent$185.732025-07-26
21Kimi K2 Thinkingthinking63.4%53.2mini-SWE-agent$219.152025-12-10
22MiniMax-M261.0%52.6mini-SWE-agent$214.132025-11-24
23DeepSeek V3.2 Reasoner60.0%–mini-SWE-agent$14.042025-12-01
24GPT-5 minimedium59.8%53.5mini-SWE-agent$17.742025-08-07
25o358.4%61.0mini-SWE-agent$166.832025-07-26
26Devstral Small 256.4%39.4mini-SWE-agent–2025-12-09
27GPT-5 mini56.2%56.4mini-SWE-agent$23.602026-02-17
28GLM-4.655.4%49.8mini-SWE-agent$48.312025-12-01
29Qwen3 Coder 480B55.4%48.8mini-SWE-agent$123.962025-08-02
30GLM-4.554.2%51.6mini-SWE-agent$148.602025-08-22
31Devstral 253.8%42.8mini-SWE-agent–2025-12-09
32Gemini 2.5 Pro53.6%54.5mini-SWE-agent$144.192025-07-26
33Claude 3.7 Sonnet52.8%49.9mini-SWE-agent$177.312025-07-20
34o4-mini45.0%56.9mini-SWE-agent$104.992025-07-26
35Kimi K243.8%51.0mini-SWE-agent$265.802025-08-07
36GPT-4.139.6%48.5mini-SWE-agent$73.162025-07-26
37GPT-5 nanomedium34.8%45.4mini-SWE-agent$19.042025-08-07
38Gemini 2.5 Flash28.7%52.6mini-SWE-agent$66.942025-07-26
39gpt-oss-120b26.0%45.5mini-SWE-agent$28.562025-08-07
40GPT-4.1 mini23.9%44.8mini-SWE-agent$217.582025-07-20
41GPT-4o21.6%43.3mini-SWE-agent$765.572025-07-20
42Llama 4 Maverick21.0%42.1mini-SWE-agent$156.852025-07-20
43Gemini 2.0 Flash13.5%43.4mini-SWE-agent–2025-07-26
44Llama 4 Scout9.1%38.6mini-SWE-agent$57.822025-07-20
45Qwen2.5 Coder 32B Instruct9.0%42.7mini-SWE-agent$34.062025-08-03

More coding benchmarks · All LLM benchmarks

Cite as: BenchLeader, “SWE-bench Verified (bash only) leaderboard”, data as of 7 Nov 2026, https://www.benchleader.com/benchmarks/swe_bench_verified.

SWE-bench Verified (bash only): questions

What does SWE-bench Verified (bash only) measure?
SWE-bench Verified is 500 real GitHub issues from popular Python repositories, each screened by human annotators to make sure the task is well specified and the tests are fair. The model reads the issue, edits the repository, and its patch is accepted only if the hidden tests pass. Scores are reported in percent of tasks solved; higher is better.
What is the top score on SWE-bench Verified (bash only)?
As of 7 Nov 2026, the highest score on SWE-bench Verified (bash only) is 76.8% by Claude Opus 4.5 (result dated 17 Feb 2026), ahead of Gemini 3 Flash at 75.8%. Scores come from SWE-bench and are re-read daily.
Which AI model leads SWE-bench Verified (bash only)?
Claude Opus 4.5 leads SWE-bench Verified (bash only) with 76.8% as of 7 Nov 2026 (result dated 17 Feb 2026), ahead of Gemini 3 Flash at 75.8%.
How many models have SWE-bench Verified (bash only) results?
45 model configurations have a SWE-bench Verified (bash only) result on BenchLeader, all taken from SWE-bench.
Who runs SWE-bench Verified (bash only) and how often is it updated?
SWE-bench Verified (bash only) is published by SWE-bench. BenchLeader re-reads the published results once a day and records the date each result was published.
Does SWE-bench Verified (bash only) count toward the BenchLeader Index?
Yes. SWE-bench Verified (bash only) contributes to the coding category of the BenchLeader Index, normalised so that 50 is the average of the evaluated models.