AllenaiAuto-detected
Olmo 3.1 32B
Best configuration ranks #410 of 610 on the BenchLeader Index at 44.6 ±7.6 (thinking reasoning effort). Last measured 2 Sept 2026.
- Blended price
- –
- Output speed
- –
- First answer
- –
- Context
- 66k
- Overall index45
- Reasoning46
- Coding46
- Agents & tools34
- Knowledge44
- Instruction following65
- Human preference44
- Long context25
- Composite38
Reasoning-effort configurations
The same model behaves differently depending on how much it is allowed to think. Each row is one setting, scored only on the benchmarks that were run at that setting. “Default” means the publisher did not say which setting was used.
| Effort | Index | Rank | Speed | First answer | Chat reply cost | Categories |
|---|---|---|---|---|---|---|
| thinkingbest | 44.6 | #410 | – | – | – | Agents & tools 34 · Coding 46 · Composite 38 · Human preference 44 · Instruction following 65 · Knowledge 44 · Long context 25 · Reasoning 46 |
| default | 43.4 | #439 | – | – | – | Agents & tools 34 · Coding 51 · Composite 37 · Human preference 50 · Instruction following 42 · Knowledge 39 · Long context 25 · Reasoning 51 |
Benchmark results
One column per reasoning effort. Rank is among every configuration of every model on that benchmark. Hover a score for the run it came from.
Reasoning
| Benchmark | thinking | default | Source |
|---|---|---|---|
| LMArena Hard Prompts | 1305#232 | 1348#195 | LMArena |
| GPQA Diamond (AA)not in index | 59.1%#362 | 53.9%#389 | Artificial Analysis |
| Humanity's Last Exam (AA)not in index | 6.3%#326 | 5.0%#377 | Artificial Analysis |
Coding
| Benchmark | thinking | default | Source |
|---|---|---|---|
| LMArena Coding | 1340#231 | 1382#191 | LMArena |
Agents & tools
| Benchmark | thinking | default | Source |
|---|---|---|---|
| Terminal-Bench Hard | 0.0%#345 | 0.0%#345 | Artificial Analysis |
| τ²-Bench Telecom (AA)not in index | 0.0%#375 | 21.4%#309 | Artificial Analysis |
Knowledge
| Benchmark | thinking | default | Source |
|---|---|---|---|
| AA-Omniscience | -43.4#275 | -52.4#335 | Artificial Analysis |
Instruction following
| Benchmark | thinking | default | Source |
|---|---|---|---|
| IFBench | 66.0%#99 | 39.2%#259 | Artificial Analysis |
Human preference
| Benchmark | thinking | default | Source |
|---|---|---|---|
| LMArena Text | 1286#245 | 1330#200 | LMArena |
Long context
| Benchmark | thinking | default | Source |
|---|---|---|---|
| AA-LCR | 0.0%#419 | 0.0%#419 | Artificial Analysis |
Composite
| Benchmark | thinking | default | Source |
|---|---|---|---|
| AA Intelligence Index | 7.1#412 | 6.5#439 | Artificial Analysis |
See also
Data as of 9 Sept 2026. Compare these configurations.