BenchLeader

WeirdML leaderboard

As of 15 Nov 2026, the top score on WeirdML is 93.6% by GPT-6 Astra (OpenAI), ahead of Claude Fable 5.1 at 92.9%, across 159 model configurations with a published result (WeirdML, result dated 3 Sep 2026). WeirdML poses unusual machine-learning problems, such as classifying odd synthetic data or learning from strange input formats, that cannot be solved with a memorised recipe. The model must write training code that actually works on held-out data. WeirdML counts toward the BenchLeader coding score.

Key facts (as of 15 Nov 2026)

  • GPT-6 Astra holds the top WeirdML score at 93.6% (WeirdML, 3 Sep 2026).
  • Claude Fable 5.1 is second on WeirdML at 92.9% (WeirdML, 1 Sep 2026).
  • WeirdML: Unusual machine-learning tasks that require writing working training code (WeirdML).
  • 159 model configurations have a published WeirdML result on BenchLeader (BenchLeader, 15 Nov 2026).
  • WeirdML feeds the coding category of the BenchLeader Index (BenchLeader methodology).

Unusual machine-learning tasks that require writing working training code.

Published by
WeirdMLdata via Epoch AI Benchmarking Hub
Category
Coding
Index weight
0.5
Models
159
Data as of
15 Nov 2026

CC BY 4.0 — Epoch AI, ‘AI Benchmarking Hub’, epoch.ai/benchmarks. Mirrored boards credit their original publishers.

What the test looks like

WeirdML poses unusual machine-learning problems, such as classifying odd synthetic data or learning from strange input formats, that cannot be solved with a memorised recipe. The model must write training code that actually works on held-out data.

How it is scored

Accuracy achieved by the model's own code on the hidden test set, averaged across tasks; run by the benchmark's author.

What to keep in mind

Small task set, and the score blends coding skill with ML judgment. Run-to-run variance is noticeable because training runs themselves vary.

159 of 159
#
1GPT-6 Astrapro mode93.6%–2026-09-03
2GPT-6 Astramax93.3%67.32026-09-03
3Claude Fable 5.1max92.9%66.72026-09-01
4GPT-6 Astrahigh92.9%66.32026-09-03
5Claude Fable 5.1high92.3%63.72026-09-01
6Claude Fable 5max91.9%65.12026-06-09
7Claude Opus 5max91.8%65.02026-07-24
8Claude Opus 5high91.6%64.42026-07-24
9GPT-5.6 Solpro mode89.4%–2026-07-09
10GPT-5.6 Solhigh88.8%63.42026-07-09
11Claude Fable 5high87.8%63.92026-06-09
12GPT-5.6 Solmax87.0%65.32026-07-09
13Claude Opus 586.3%63.22026-07-24
14GPT-5.5xhigh84.9%64.12026-04-23
15Gemini 3.8 Flashhigh84.8%59.42026-09-02
16GPT-5.5high83.9%63.52026-04-23
17Claude Opus 4.8xhigh82.9%–2026-05-28
18Kimi K3max82.6%60.22026-07-16
19GPT-5.3-Codex79.3%–2026-02-05
20GPT-5.6 Terrahigh78.3%56.82026-07-09
21Claude Opus 4.6high78.0%62.52026-02-05
22GPT-5.3-Codexxhigh77.9%62.92026-02-05
23Claude Opus 4.677.9%62.12026-02-05
24GPT-5.4xhigh77.7%63.12026-03-05
25Claude Opus 4.7high76.4%60.02026-04-16
26Claude Opus 4.776.4%63.12026-04-16
27Claude Opus 4.8medium76.0%–2026-05-28
28Claude Opus 4.7max75.5%59.72026-04-16
29GLM 5.3max75.4%58.82026-08-14
30GPT-5.2xhigh72.2%59.92025-12-11
31Gemini 3.1 Pro72.1%59.92026-02-19
32Claude Opus 4.8no reasoning70.5%–2026-05-28
33GLM 5.2max70.1%58.92026-06-16
34Gemini 3 Pro69.9%60.02025-11-18
35Claude Sonnet 5high68.8%59.02026-06-30
36GLM 5.2high67.3%–2026-06-16
37Grok 4.6high67.3%60.62026-08-12
38GPT-5.5no reasoning67.2%52.32026-04-23
39DeepSeek V4 Promax66.2%54.72026-08-13
40Claude Sonnet 4.6medium66.1%–2026-02-17
41Claude Opus 4.563.7%57.12025-11-24
42GPT-5.2medium63.4%56.22025-12-11
43DeepSeek V4 Flashmax63.0%56.82026-07-31
44Gemini 3.5 Flashhigh62.6%62.52026-05-19
45Gemini 3 Flash61.6%56.52025-12-17
46GPT-5.6 Lunahigh60.9%54.72026-07-09
47GPT-5.1high60.8%57.52025-11-13
48GPT-5high60.7%58.22025-08-07
49GPT-5 Prohigh60.4%–2025-10-07
50Muse Spark 1.2xhigh60.3%61.52026-08-05
51GPT-5.4 Minihigh60.3%52.02026-03-17
52o3-prohigh58.2%55.92025-06-10
53GPT-5.4no reasoning57.4%53.02026-03-05
54GPT-5.4 Prono reasoning57.4%–2026-03-05
55GLM 5.157.1%55.42026-04-07
56DeepSeek V4 Flashhigh57.0%57.22026-07-31
57Gemini 3.6 Flashhigh56.1%61.02026-07-21
58Kimi K2.655.9%55.12026-04-20
59GPT-5-Codexhigh54.5%58.62025-09-15
60GPT-5-Codex54.5%–2025-09-15

AI coding benchmark leaderboard · All LLM benchmarks

Cite as: BenchLeader, “WeirdML leaderboard”, data as of 15 Nov 2026, https://www.benchleader.com/benchmarks/weirdml.

WeirdML: questions

What does WeirdML measure?
WeirdML poses unusual machine-learning problems, such as classifying odd synthetic data or learning from strange input formats, that cannot be solved with a memorised recipe. The model must write training code that actually works on held-out data. Scores are reported in percent of tasks solved; higher is better.
What is the top score on WeirdML?
As of 15 Nov 2026, the highest score on WeirdML is 93.6% by GPT-6 Astra (result dated 3 Sep 2026), ahead of Claude Fable 5.1 at 92.9%. Scores come from WeirdML and are re-read daily.
Which AI model leads WeirdML?
GPT-6 Astra leads WeirdML with 93.6% as of 15 Nov 2026 (result dated 3 Sep 2026), ahead of Claude Fable 5.1 at 92.9%.
How many models have WeirdML results?
159 model configurations have a WeirdML result on BenchLeader, all taken from WeirdML via Epoch AI Benchmarking Hub.
Who runs WeirdML and how often is it updated?
WeirdML is published by WeirdML. BenchLeader re-reads the published results once a day and records the date each result was published.
Does WeirdML count toward the BenchLeader Index?
Yes. WeirdML contributes to the coding category of the BenchLeader Index, normalised so that 50 is the average of the evaluated models.