BenchLeader
Menu
Best LLMsChoose a modelLeaderboardsModelsBenchmarksProvidersReleases9

SWE-Bench Pro

Long-horizon software engineering tasks in public repositories. Scale AI.

As of 25 Oct 2026, Muse Spark 1.1 leads SWE-Bench Pro on BenchLeader with 61.5%, ahead of GPT-5.4 at 59.1%, across 24 model configurations with a published result.

Published by
Scale AI SEAL
Category
Coding
Index weight
1.0
Models
24
Data as of
25 Oct 2026
Newest result
9 Jul 2026

The publisher has posted no new results on this board since 9 Jul 2026. BenchLeader re-reads it every day, so newer models appear here as soon as they are published.

Scale AI SEAL Leaderboards.

What the test looks like

SWE-Bench Pro is Scale AI's harder, contamination-resistant successor to SWE-bench: long-horizon issues from public repositories in several languages, with tasks that require multi-file changes.

How it is scored

Percent of issues resolved in Scale's harness, on the public set.

What to keep in mind

Much harder than SWE-bench Verified, so scores are lower and not comparable to it. A held-out commercial set exists that only Scale can run.

Top 15
  1. 1Muse Spark 1.161.5%
  2. 2GPT-5.4 (xhigh)59.1%
  3. 3Muse Spark55.0%
  4. 4Claude Opus 4.6 (thinking)51.9%
  5. 5Gemini 3.1 Pro (thinking)46.1%
  6. 6Claude Opus 4.545.9%
  7. 7Claude Sonnet 4.543.6%
  8. 8Gemini 3 Pro43.3%
  9. 9Claude Sonnet 442.7%
  10. 10GPT-5 (high)41.8%
  11. 11GPT-5.2-Codex41.0%
  12. 12Claude Haiku 4.539.5%
  13. 13Qwen3 Coder 480B38.7%
  14. 14Minimax 2.136.8%
  15. 15Gemini 3 Flash34.6%
24 of 24
#
1Muse Spark 1.161.5%62.52026-07-09
2GPT-5.4xhigh59.1%64.72026-04-08
3Muse Spark55.0%64.32026-04-08
4Claude Opus 4.6thinking51.9%62.52026-04-08
5Gemini 3.1 Prothinking46.1%–2026-04-08
6Claude Opus 4.545.9%57.92025-12-11
7Claude Sonnet 4.543.6%53.72025-09-19
8Gemini 3 Pro43.3%60.42025-11-26
9Claude Sonnet 442.7%50.52025-09-19
10GPT-5high41.8%57.92025-11-26
11GPT-5.2-Codex41.0%57.32026-01-27
12Claude Haiku 4.539.5%45.32025-09-19
13Qwen3 Coder 480B38.7%48.02026-01-27
14Minimax 2.136.8%–2026-01-12
15Gemini 3 Flash34.6%57.02026-01-12
16GPT-5.229.9%58.02026-01-27
17Kimi K227.7%50.42025-09-19
18Qwen3 235B A22B21.4%50.12026-01-27
19gpt-oss-120b16.2%45.62026-01-27
20Gemma 3 27B11.4%38.92026-01-27
21Llama 3.1 405B11.2%42.92026-01-27
22GLM-4.69.7%49.92026-01-27
23Llama 4 Maverick5.2%42.82026-01-27
24Codestral 24051.5%–2026-01-27

More coding benchmarks · All LLM benchmarks

Cite as: BenchLeader, “SWE-Bench Pro leaderboard”, data as of 25 Oct 2026, https://www.benchleader.com/benchmarks/scale_swe_bench_pro.

SWE-Bench Pro: questions

What does SWE-Bench Pro measure?
SWE-Bench Pro is Scale AI's harder, contamination-resistant successor to SWE-bench: long-horizon issues from public repositories in several languages, with tasks that require multi-file changes. Scores are reported in percent of tasks solved; higher is better.
Which AI model leads SWE-Bench Pro?
Muse Spark 1.1 leads SWE-Bench Pro with 61.5% as of 25 Oct 2026, ahead of GPT-5.4 at 59.1%.
How many models have SWE-Bench Pro results?
24 model configurations have a SWE-Bench Pro result on BenchLeader, all taken from Scale AI SEAL.
Who runs SWE-Bench Pro and how often is it updated?
SWE-Bench Pro is published by Scale AI SEAL. BenchLeader re-reads the published results once a day and records the date each result was published.
Does SWE-Bench Pro count toward the BenchLeader Index?
Yes. SWE-Bench Pro contributes to the coding category of the BenchLeader Index, normalised so that 50 is the average of the evaluated models.