BenchLeader
Menu
Best LLMsChoose a modelLeaderboardsModelsBenchmarksProvidersReleases12

WeirdML

Unusual machine-learning tasks that require writing working training code.

As of 24 Oct 2026, GPT-6 Astra Pro leads WeirdML on BenchLeader with 93.6%, ahead of GPT-6 Astra at 93.3%, across 156 model configurations with a published result.

Published by
WeirdMLdata via Epoch AI Benchmarking Hub
Category
Coding
Index weight
0.5
Models
156
Data as of
24 Oct 2026
Newest result
3 Sept 2026

CC BY 4.0 — Epoch AI, ‘AI Benchmarking Hub’, epoch.ai/benchmarks. Mirrored boards credit their original publishers.

What the test looks like

WeirdML poses unusual machine-learning problems, such as classifying odd synthetic data or learning from strange input formats, that cannot be solved with a memorised recipe. The model must write training code that actually works on held-out data.

How it is scored

Accuracy achieved by the model's own code on the hidden test set, averaged across tasks; run by the benchmark's author.

What to keep in mind

Small task set, and the score blends coding skill with ML judgment. Run-to-run variance is noticeable because training runs themselves vary.

Top 15
  1. 1GPT-6 Astra Pro (max)93.6%
  2. 2GPT-6 Astra (max)93.3%
  3. 3Claude Fable 5.1 (max)92.9%
  4. 4GPT-6 Astra (high)92.9%
  5. 5Claude Fable 5.1 (high)92.3%
  6. 6Claude Fable 5 (max)91.9%
  7. 7Claude Opus 5 (max)91.8%
  8. 8Claude Opus 5 (high)91.6%
  9. 9GPT-5.6 Sol Pro (max)89.4%
  10. 10GPT-5.6 Sol (high)88.8%
  11. 11Claude Fable 5 (high)87.8%
  12. 12GPT-5.6 Sol (max)87.0%
  13. 13Claude Opus 586.3%
  14. 14GPT-5.5 (xhigh)84.9%
  15. 15Gemini 3.8 Flash (high)84.8%
156 of 156
#
1GPT-6 Astra Promax93.6%–2026-09-03
2GPT-6 Astramax93.3%70.42026-09-03
3Claude Fable 5.1max92.9%70.52026-09-01
4GPT-6 Astrahigh92.9%69.62026-09-03
5Claude Fable 5.1high92.3%69.42026-09-01
6Claude Fable 5max91.9%69.32026-06-09
7Claude Opus 5max91.8%68.82026-07-24
8Claude Opus 5high91.6%67.92026-07-24
9GPT-5.6 Sol Promax89.4%–2026-07-09
10GPT-5.6 Solhigh88.8%66.32026-07-09
11Claude Fable 5high87.8%64.62026-06-09
12GPT-5.6 Solmax87.0%68.02026-07-09
13Claude Opus 586.3%65.52026-07-24
14GPT-5.5xhigh84.9%66.12026-04-23
15Gemini 3.8 Flashhigh84.8%64.42026-09-02
16GPT-5.5high83.9%65.32026-04-23
17Claude Opus 4.8xhigh82.9%–2026-05-28
18Kimi K3max82.6%64.72026-07-16
19GPT-5.3-Codex79.3%–2026-02-05
20GPT-5.6 Terrahigh78.3%61.32026-07-09
21Claude Opus 4.6high78.0%62.72026-02-05
22GPT-5.3-Codexxhigh77.9%63.82026-02-05
23Claude Opus 4.677.9%62.62026-02-05
24GPT-5.4xhigh77.7%64.72026-03-05
25Claude Opus 4.7high76.4%61.02026-04-16
26Claude Opus 4.776.4%63.32026-04-16
27Claude Opus 4.8medium76.0%–2026-05-28
28Claude Opus 4.7max75.5%62.42026-04-16
29GLM 5.3max75.4%63.52026-08-14
30GPT-5.2xhigh72.2%60.72025-12-11
31Gemini 3.1 Pro72.1%60.62026-02-19
32Claude Opus 4.8no reasoning70.5%–2026-05-28
33GLM 5.2max70.1%62.32026-06-16
34Gemini 3 Pro69.9%60.42025-11-18
35Claude Sonnet 5high68.8%60.22026-06-30
36GLM 5.2high67.3%–2026-06-16
37Grok 4.6high67.3%62.92026-08-12
38GPT-5.5no reasoning67.2%52.82026-04-23
39DeepSeek V4 Promax66.2%60.82026-08-13
40Claude Sonnet 4.6medium66.1%–2026-02-17
41Claude Opus 4.563.7%57.92025-11-24
42GPT-5.2medium63.4%57.62025-12-11
43DeepSeek V4 Flashmax63.0%59.42026-07-31
44Gemini 3.5 Flashhigh62.6%63.12026-05-19
45Gemini 3 Flash61.6%57.02025-12-17
46GPT-5.6 Lunahigh60.9%57.02026-07-09
47GPT-5.1high60.8%57.72025-11-13
48GPT-5high60.7%57.92025-08-07
49GPT-5 Prohigh60.4%–2025-10-07
50Muse Spark 1.2xhigh60.3%63.22026-08-05
51GPT-5.4 Minihigh60.3%52.22026-03-17
52o3-prohigh58.2%53.72025-06-10
53GPT-5.4no reasoning57.4%52.62026-03-05
54GPT-5.4 Prono reasoning57.4%–2026-03-05
55GLM 5.157.1%55.92026-04-07
56DeepSeek V4 Flashhigh57.0%57.42026-07-31
57Gemini 3.6 Flashhigh56.1%62.02026-07-21
58Kimi K2.655.9%55.42026-04-20
59GPT-5-Codexhigh54.5%57.52025-09-15
60GPT-5-Codex54.5%–2025-09-15

More coding benchmarks · All LLM benchmarks

Cite as: BenchLeader, “WeirdML leaderboard”, data as of 24 Oct 2026, https://www.benchleader.com/benchmarks/weirdml.

WeirdML: questions

What does WeirdML measure?
WeirdML poses unusual machine-learning problems, such as classifying odd synthetic data or learning from strange input formats, that cannot be solved with a memorised recipe. The model must write training code that actually works on held-out data. Scores are reported in percent of tasks solved; higher is better.
Which AI model leads WeirdML?
GPT-6 Astra Pro leads WeirdML with 93.6% as of 24 Oct 2026, ahead of GPT-6 Astra at 93.3%.
How many models have WeirdML results?
156 model configurations have a WeirdML result on BenchLeader, all taken from WeirdML via Epoch AI Benchmarking Hub.
Who runs WeirdML and how often is it updated?
WeirdML is published by WeirdML. BenchLeader re-reads the published results once a day and records the date each result was published.
Does WeirdML count toward the BenchLeader Index?
Yes. WeirdML contributes to the coding category of the BenchLeader Index, normalised so that 50 is the average of the evaluated models.