BenchLeader
Menu
Best LLMsChoose a modelCoding plansLeaderboardsModelsBenchmarksProvidersReleases7

ITBench SRE (AA) leaderboard

As of 6 Nov 2026, the top score on ITBench SRE (AA) is 56.2% by GPT-5.6 Sol (OpenAI), ahead of Step 5 Preview at 55.6%, across 45 model configurations with a published result (Artificial Analysis, result dated 9 Jul 2026). ITBench SRE drops an agent into a running IT environment with an incident to diagnose and fix, in the style of on-call engineering work. ITBench SRE (AA) is shown for reference only and does not count toward the BenchLeader Index.

Key facts (as of 6 Nov 2026)

  • GPT-5.6 Sol holds the top ITBench SRE (AA) score at 56.2% (Artificial Analysis, 9 Jul 2026).
  • Step 5 Preview is second on ITBench SRE (AA) at 55.6% (Artificial Analysis, 18 Sep 2026).
  • ITBench SRE (AA): Site-reliability incident tasks in a live IT environment, run by Artificial Analysis (Artificial Analysis).
  • 45 model configurations have a published ITBench SRE (AA) result on BenchLeader (BenchLeader, 6 Nov 2026).
  • ITBench SRE (AA) does not feed the BenchLeader Index; it is shown for reference (BenchLeader methodology).

Site-reliability incident tasks in a live IT environment, run by Artificial Analysis.

Published by
Artificial Analysis
Category
Agents & tools
Index weight
Reference only
Models
45
Data as of
6 Nov 2026

Source: Artificial Analysis (artificialanalysis.ai). Data taken from the public leaderboard.

What the test looks like

ITBench SRE drops an agent into a running IT environment with an incident to diagnose and fix, in the style of on-call engineering work.

How it is scored

Percent of incidents resolved, run by Artificial Analysis.

What to keep in mind

Few models have results yet and each task is long, so the score is volatile.

Top 15
  1. 1GPT-5.6 Sol (max)56.2%
  2. 2Step 5 Preview55.6%
  3. 3Gemini 3.8 Flash (high)52.5%
  4. 4GLM 5.3 Flash51.2%
  5. 5GPT-5.6 Terra (max)51.0%
  6. 6Claude Fable 5.1 (max)49.5%
  7. 7GPT-6 Sol (max)49.4%
  8. 8GPT-6 Astra (max)48.6%
  9. 9Kimi K3 (max)47.7%
  10. 10DeepSeek V4.1 Flash (max)46.9%
  11. 11Claude Opus 4.7 (max)46.7%
  12. 12GLM 5.3 (max)46.1%
  13. 13GPT-5.5 (xhigh)45.8%
  14. 14GLM 5.2 (max)42.7%
  15. 15Qwen3.7 (max)42.5%
45 of 45
#
1GPT-5.6 Solmax56.2%65.42026-07-09
2Step 5 Preview55.6%54.92026-09-18
3Gemini 3.8 Flashhigh52.5%62.02026-09-02
4GLM 5.3 Flash51.2%56.82026-08-26
5GPT-5.6 Terramax51.0%60.72026-07-09
6Claude Fable 5.1max49.5%67.22026-09-01
7GPT-6 Solmax49.4%63.32026-09-22
8GPT-6 Astramax48.6%67.82026-09-03
9Kimi K3max47.7%62.12026-07-16
10DeepSeek V4.1 Flashmax46.9%59.02026-09-10
11Claude Opus 4.7max46.7%59.52026-04-16
12GLM 5.3max46.1%59.02026-08-18
13GPT-5.5xhigh45.8%64.32026-04-23
14GLM 5.2max42.7%59.52026-06-16
15Qwen3.7max42.5%60.32026-05-19
16Grok 4.7xhigh42.1%58.32026-09-21
17Gemini 3.5 Flashhigh40.4%62.52026-05-19
18GPT-5.6 Lunamax40.3%56.32026-07-09
19Qwen3.8 Maxmax40.3%58.82026-09-02
20GLM 5.1thinking40.3%–2026-04-07
21Claude Sonnet 4.6max39.8%54.22026-02-17
22DeepSeek V4 Pro 0424max38.3%–2026-04-24
23Claude Opus 5.5max38.2%69.32026-09-22
24MiMo-V2.5-Prothinking38.2%–2026-04-22
25Gemma 4 31Bthinking37.3%46.52026-04-02
26Qwen3.5-27Bthinking35.5%–2026-02-24
27GPT-5.4 Minixhigh35.2%56.02026-03-17
28Qwen3.5 397B A17Bthinking34.1%56.62026-02-16
29Muse Spark 1.3max33.2%62.02026-09-02
30Grok 4.3high32.7%56.62026-04-30
31DeepSeek V4 Flash 0420max31.5%–2026-04-24
32Kimi K2.6thinking31.2%59.72026-04-20
33Gemini 3.1 Pro30.3%60.52026-02-19
34Step 3.7 Flash30.3%53.02026-05-29
35Claude Haiku 4.5thinking27.3%45.42025-10-15
36MiniMax M2.726.5%56.52026-03-18
37GPT-5.4 Nanoxhigh24.4%54.02026-03-17
38Gemma 4 26B A4Bthinking23.6%51.02026-04-02
39Qwen3.5-35B-A3Bthinking21.5%–2026-02-24
40GPT-5.4 Minino reasoning18.6%44.82026-03-17
41Grok 4.1 Fastno reasoning17.9%43.52025-11-19
42GPT-5.4 Nanono reasoning12.9%41.82026-03-17
43gpt-oss-120bhigh5.7%48.92025-08-05
44Nemotron 3 Super 120B A12Bthinking1.1%51.12026-03-11
45Llama 3.3 70B0.6%40.02024-12-06

More agents & tools benchmarks · All LLM benchmarks

Related benchmarks

Top head-to-heads on ITBench SRE (AA)

Cite as: BenchLeader, “ITBench SRE (AA) leaderboard”, data as of 6 Nov 2026, https://www.benchleader.com/benchmarks/aa_itbench_sre.

ITBench SRE (AA): questions

What does ITBench SRE (AA) measure?
ITBench SRE drops an agent into a running IT environment with an incident to diagnose and fix, in the style of on-call engineering work. Scores are reported in percent of tasks solved; higher is better.
What is the top score on ITBench SRE (AA)?
As of 6 Nov 2026, the highest score on ITBench SRE (AA) is 56.2% by GPT-5.6 Sol (result dated 9 Jul 2026), ahead of Step 5 Preview at 55.6%. Scores come from Artificial Analysis and are re-read daily.
Which AI model leads ITBench SRE (AA)?
GPT-5.6 Sol leads ITBench SRE (AA) with 56.2% as of 6 Nov 2026 (result dated 9 Jul 2026), ahead of Step 5 Preview at 55.6%.
How many models have ITBench SRE (AA) results?
45 model configurations have a ITBench SRE (AA) result on BenchLeader, all taken from Artificial Analysis.
Who runs ITBench SRE (AA) and how often is it updated?
ITBench SRE (AA) is published by Artificial Analysis. BenchLeader re-reads the published results once a day and records the date each result was published.
Does ITBench SRE (AA) count toward the BenchLeader Index?
No. ITBench SRE (AA) is shown for reference but left out of the composite index.