HiL-Bench
Whether agents notice information gaps and ask clarifying questions. Scale AI.
As of 24 Oct 2026, Muse Wonder leads HiL-Bench on BenchLeader with 79.7%, ahead of Muse Spark 1.3 at 66.9%, across 30 model configurations with a published result.
- Published by
- Scale AI SEAL
- Category
- Agents & tools
- Index weight
- 0.5
- Models
- 30
- Data as of
- 24 Oct 2026
- Newest result
- 21 Oct 2026
Scale AI SEAL Leaderboards.
What the test looks like
HiL-Bench checks whether an agent notices when a task is under-specified and asks a clarifying question instead of guessing.
How it is scored
Percent of cases handled correctly, published by Scale AI.
What to keep in mind
A behavioural test that rewards asking; models tuned to always act score poorly even when capable.
- 1Muse Wonder79.7%
- 2Muse Spark 1.366.9%
- 3GPT-6 Astra65.3%
- 4GPT-6.1 Sol62.3%
- 5Gemini 4 Argon62.0%
- 6Claude Fable 5.161.5%
- 7SWE-261.5%
- 8Muse Spark Open59.7%
- 9Claude Opus 5.559.0%
- 10GLM 5.358.3%
- 11Claude Opus 557.0%
- 12Claude Fable 556.3%
- 13DeepSeek V4 Flash47.8%
- 14GLM 5.243.7%
- 15Kimi K342.7%
30 of 30
| # | ||||
|---|---|---|---|---|
| 1 |  Muse Wonder | 79.7% | 63.7 | 2026-10-14 |
| 2 | Muse Spark 1.3 | 66.9% | 59.8 | 2026-10-13 |
| 3 |  GPT-6 Astra | 65.3% | 68.0 | 2026-10-13 |
| 4 | GPT-6.1 Sol | 62.3% | 63.0 | 2026-10-13 |
| 5 |  Gemini 4 Argon | 62.0% | 64.9 | 2026-10-16 |
| 6 |  Claude Fable 5.1 | 61.5% | 66.4 | 2026-09-03 |
| 7 |  SWE-2 | 61.5% | – | 2026-10-21 |
| 8 | Muse Spark Openopen ↗ | 59.7% | – | 2026-10-20 |
| 9 | Claude Opus 5.5 | 59.0% | 62.0 | 2026-10-13 |
| 10 |  GLM 5.3open ↗ | 58.3% | – | 2026-10-13 |
| 11 | Claude Opus 5 | 57.0% | 65.5 | 2026-07-28 |
| 12 | Claude Fable 5 | 56.3% | 67.1 | 2026-07-20 |
| 13 |  DeepSeek V4 Flashopen ↗ | 47.8% | 52.2 | 2026-10-13 |
| 14 | GLM 5.2open ↗ | 43.7% | 52.0 | 2026-07-20 |
| 15 |  Kimi K3open ↗ | 42.7% | 59.1 | 2026-10-13 |
| 16 | Claude Opus 4.7 | 41.7% | 63.3 | 2026-04-17 |
| 17 | Gemini 3.8 Flash | 41.5% | 55.7 | 2026-09-09 |
| 18 | GPT-5.5 | 39.7% | 60.3 | 2026-04-29 |
| 19 | DeepSeek V4 Proopen ↗ | 38.5% | 53.5 | 2026-10-13 |
| 20 | Claude Opus 4.6 | 38.3% | 62.6 | 2026-04-16 |
| 21 |  MiniMax M3open ↗ | 38.3% | 56.4 | 2026-10-13 |
| 22 | Gemini 3.1 Pro | 35.3% | 60.6 | 2026-04-16 |
| 23 | Claude Opus 4.8 | 35.3% | 60.1 | 2026-05-30 |
| 24 | GPT-5.6 Sol | 32.3% | 55.9 | 2026-07-20 |
| 25 | Gemini 3.5 Flash | 27.7% | 49.1 | 2026-05-30 |
| 26 |  Grok 4.20 | 20.0% | – | 2026-04-16 |
| 27 | Kimi K2.6open ↗ | 18.7% | 55.4 | 2026-04-29 |
| 28 | GPT-5.4 | 9.7% | 58.2 | 2026-04-16 |
| 29 | MiniMax M2.5open ↗ | 6.3% | 53.6 | 2026-04-16 |
| 30 | GPT-5.3-Codex | 4.3% | – | 2026-04-16 |
More agents & tools benchmarks · All LLM benchmarks
Cite as: BenchLeader, “HiL-Bench leaderboard”, data as of 24 Oct 2026, https://www.benchleader.com/benchmarks/scale_hil.
HiL-Bench: questions
- What does HiL-Bench measure?
- HiL-Bench checks whether an agent notices when a task is under-specified and asks a clarifying question instead of guessing. Scores are reported in percent of tasks solved; higher is better.
- Which AI model leads HiL-Bench?
- Muse Wonder leads HiL-Bench with 79.7% as of 24 Oct 2026, ahead of Muse Spark 1.3 at 66.9%.
- How many models have HiL-Bench results?
- 30 model configurations have a HiL-Bench result on BenchLeader, all taken from Scale AI SEAL.
- Who runs HiL-Bench and how often is it updated?
- HiL-Bench is published by Scale AI SEAL. BenchLeader re-reads the published results once a day and records the date each result was published.
- Does HiL-Bench count toward the BenchLeader Index?
- Yes. HiL-Bench contributes to the agents & tools category of the BenchLeader Index, normalised so that 50 is the average of the evaluated models.