FrontierMath tier 4
Mathematics · fixed question set
The hardest FrontierMath tier: problems aimed at research mathematicians.
How far to trust it
Scores are low and the set is small, so a few problems move the number a long way. Treat small gaps as noise.
Measured
Models scored
64
Spread between models
16.3
standard deviation, points
Measurement noise
3.5
published stderr
Weight
0.95
share of spread that is signal
The weight is (spread² − noise²) / spread². It is computed, never chosen. Most of this benchmark's spread is real, so it still separates models.
Every model scored
the median across configurations, so one heroic run cannot lead
| Rank | Model | Lab | Score |
|---|---|---|---|
| 1 | Claude Fable 5 | Anthropic | 100.0 |
| 2 | AI Co-Mathematician | 47.9 | |
| 3 | GPT-5.4 | OpenAI | 44.3 |
| 4 | GPT-5.5 Pro pre-release | OpenAI | 39.6 |
| 5 | GPT-5.4 Pro | OpenAI | 37.5 |
| 6 | GPT-5.5 | OpenAI | 35.4 |
| 7 | GPT-5.2 Pro | OpenAI | 31.3 |
| 8 | Claude Opus 4.8 | Anthropic | 31.2 |
| 9 | GPT-5 Pro | OpenAI | 14.6 |
| 10 | Muse Spark | Meta | 14.6 |
| 11 | Claude Opus 4.6 | Anthropic | 14.6 |
| 12 | Kimi K2.6 | Kimi | 14.6 |
| 13 | Claude Opus 4.7 | Anthropic | 11.5 |
| 14 | Claude Opus 4.6 | Anthropic | 10.4 |
| 15 | Gemini 2.5 Deep Think | 10.4 | |
| 16 | Gemini 3 Pro Preview | 9.4 | |
| 17 | GPT 5.2 | OpenAI | 8.9 |
| 18 | Gemini 3.1 Pro Preview | 8.3 | |
| 19 | GPT 5.1 | OpenAI | 8.3 |
| 20 | Gemini 3.5 Flash | 7.3 | |
| 21 | GLM-5.1 | Zhipu | 6.2 |
| 22 | GPT-5.4 nano | OpenAI | 6.2 |
| 23 | GPT 5 | OpenAI | 4.7 |
| 24 | Kimi K2.5 | Kimi | 4.2 |
| 25 | Claude Opus 4.1 | Anthropic | 4.2 |
| 26 | Claude Opus 4.5 | Anthropic | 4.2 |
| 27 | Claude Sonnet 4.5 | Anthropic | 4.2 |
| 28 | Gemini 2.5 Flash | 4.2 | |
| 29 | Qwen3.6 Plus | Alibaba | 4.2 |
| 30 | Claude Sonnet 4.6 | Anthropic | 4.2 |
| 31 | Claude Opus 4.5 | Anthropic | 3.1 |
| 32 | GPT 5 mini | OpenAI | 2.6 |
| 33 | DeepSeek-V3.2-Exp | DeepSeek | 2.1 |
| 34 | Gemini 2.5 Pro | 2.1 | |
| 35 | Gemini 3 Flash Preview | 2.1 | |
| 36 | O3 Mini | OpenAI | 2.1 |
| 37 | Qwen3.6 Max Preview | Alibaba | 2.1 |
| 38 | o4-mini | OpenAI | 2.1 |
| 39 | Claude Haiku 4.5 | Anthropic | 2.1 |
| 40 | GPT-5.4 mini | OpenAI | 2.1 |
| 41 | Grok 4 Heavy | xAI | 2.1 |
| 42 | Qwen3.5 Plus | Alibaba | 2.1 |
| 43 | GLM 4.6 | Zhipu | 1.1 |
| 44 | GLM 5 | Zhipu | 1.1 |
| 45 | Claude Opus 4 | Anthropic | 1.0 |
| 46 | Claude Sonnet 4.5 | Anthropic | 1.0 |
| 47 | Gemini 2.5 Pro Preview | 1.0 | |
| 48 | Grok 4 | xAI | 1.0 |
| 49 | o3 | OpenAI | 1.0 |
| 50 | GPT 5 nano | OpenAI | 0.5 |
| 51 | Claude 3.5 Sonnet | Anthropic | 0.0 |
| 52 | Claude 3.5 Sonnet | Anthropic | 0.0 |
| 53 | Claude 3.7 Sonnet | Anthropic | 0.0 |
| 54 | Claude Sonnet 4 | Anthropic | 0.0 |
| 55 | DeepSeek-R1 | DeepSeek | 0.0 |
| 56 | GLM 4.5 | Zhipu | 0.0 |
| 57 | GLM-4.7 | Zhipu | 0.0 |
| 58 | GPT 4.1 | OpenAI | 0.0 |
| 59 | Grok 3 Beta | xAI | 0.0 |
| 60 | Grok-3 mini | xAI | 0.0 |
| 61 | Kimi K2 Instruct | Kimi | 0.0 |
| 62 | Qwen3-235B-A22B | Alibaba | 0.0 |
| 63 | Qwen3.5 Flash | Alibaba | 0.0 |
| 64 | Qwen3.6 Flash | Alibaba | 0.0 |