BenchLeader
Menu
Best LLMsChoose a modelLeaderboardsModelsBenchmarksProvidersReleases9

MultiChallenge

Multi-turn instruction following under realistic conversation constraints. Scale AI.

As of 25 Oct 2026, GPT-6.1 Sol leads MultiChallenge on BenchLeader with 82.0%, ahead of Gemini 4 Argon at 78.7%, across 33 model configurations with a published result.

Published by
Scale AI SEAL
Category
Instruction following
Index weight
1.0
Models
33
Data as of
25 Oct 2026
Newest result
21 Oct 2026

Scale AI SEAL Leaderboards.

What the test looks like

MultiChallenge tests instruction following across a multi-turn conversation: keeping earlier constraints, tracking what the user changed, and not contradicting itself.

How it is scored

Percent of conversations where all constraints are met, judged by a model-graded rubric with human validation, published by Scale AI.

What to keep in mind

Model-graded, so grading quality matters; conversations are synthetic.

Top 15
  1. 1GPT-6.1 Sol82.0%
  2. 2Gemini 4 Argon78.7%
  3. 3Muse Spark75.5%
  4. 4Muse Spark 1.175.3%
  5. 5Muse Spark Open74.2%
  6. 6Gemini 3.1 Pro71.4%
  7. 7GPT-5.4 Pro69.2%
  8. 8Claude Haiku 5.567.7%
  9. 9Gemini 3 Pro65.7%
  10. 10GPT-5.1 (thinking)63.4%
  11. 11GPT-5 (thinking)63.2%
  12. 12o3-pro (high)62.4%
  13. 13Kimi K2.561.4%
  14. 14Gemini 3.1 Flash Lite60.6%
  15. 15GPT-5 mini (thinking)59.0%
33 of 33
#
1GPT-6.1 Sol82.0%63.02026-10-07
2Gemini 4 Argon78.7%63.62026-10-21
3Muse Spark75.5%64.32026-10-07
4Muse Spark 1.175.3%62.52026-10-07
5Muse Spark Open74.2%–2026-10-21
6Gemini 3.1 Pro71.4%60.52026-10-07
7GPT-5.4 Pro69.2%63.32026-10-07
8Claude Haiku 5.567.7%–2026-10-08
9Gemini 3 Pro65.7%60.42026-10-07
10GPT-5.1thinking63.4%54.12026-10-07
11GPT-5thinking63.2%–2026-10-07
12o3-prohigh62.4%53.72026-10-07
13Kimi K2.561.4%54.52026-10-07
14Gemini 3.1 Flash Lite60.6%53.72026-10-07
15GPT-5 minithinking59.0%–2026-10-07
16Claude Opus 4.5thinking59.0%59.92026-10-07
17Claude Opus 4thinking58.6%55.02026-10-07
18Claude Opus 4.1thinking57.2%56.42026-10-07
19Claude Sonnet 4thinking57.1%53.22026-10-07
20o3high56.6%53.92026-10-07
21Claude Opus 4.6no reasoning56.0%56.52026-10-07
22Kimi K2 Thinkingthinking55.4%52.42026-10-07
23Claude Sonnet 4.5thinking55.3%52.62026-10-07
24Gemini 2.5 Pro53.6%53.92026-10-07
25Claude 3.7 Sonnetthinking51.6%51.02026-10-07
26GPT 5.1 Instant51.2%–2026-10-07
27Claude Haiku 4.5thinking50.5%46.72026-10-07
28gpt-oss-120b45.3%45.62026-10-07
29o4-minihigh44.9%50.12026-10-07
30Qwen3 235B A22B41.2%50.12026-10-07
31GPT-4.139.4%48.32026-10-07
32Claude Opus 4.6max37.1%57.42026-10-07
33Gemini 2.0 Flash36.4%43.22026-10-07

More instruction following benchmarks · All LLM benchmarks

Cite as: BenchLeader, “MultiChallenge leaderboard”, data as of 25 Oct 2026, https://www.benchleader.com/benchmarks/scale_multichallenge.

MultiChallenge: questions

What does MultiChallenge measure?
MultiChallenge tests instruction following across a multi-turn conversation: keeping earlier constraints, tracking what the user changed, and not contradicting itself. Scores are reported in percent of tasks solved; higher is better.
Which AI model leads MultiChallenge?
GPT-6.1 Sol leads MultiChallenge with 82.0% as of 25 Oct 2026, ahead of Gemini 4 Argon at 78.7%.
How many models have MultiChallenge results?
33 model configurations have a MultiChallenge result on BenchLeader, all taken from Scale AI SEAL.
Who runs MultiChallenge and how often is it updated?
MultiChallenge is published by Scale AI SEAL. BenchLeader re-reads the published results once a day and records the date each result was published.
Does MultiChallenge count toward the BenchLeader Index?
Yes. MultiChallenge contributes to the instruction following category of the BenchLeader Index, normalised so that 50 is the average of the evaluated models.