FrontierMath tier 4 v2
Mathematics · fixed question set
The second version of the hardest FrontierMath tier.
How far to trust it
As tier 4. The v2 problems are newer, so contamination is lower still, and the two versions are not directly comparable.
Measured
Models scored
58
Spread between models
27.3
standard deviation, points
Measurement noise
6.5
published stderr
Weight
0.94
share of spread that is signal
The weight is (spread² − noise²) / spread². It is computed, never chosen. Most of this benchmark's spread is real, so it still separates models.
Every model scored
the median across configurations, so one heroic run cannot lead
| Rank | Model | Lab | Score |
|---|---|---|---|
| 1 | GPT-6 Astra | OpenAI | 97.6 |
| 2 | Claude Fable 5 | Anthropic | 90.2 |
| 3 | Claude Fable 5.1 | Anthropic | 87.8 |
| 4 | GPT-5.6 Sol | OpenAI | 82.9 |
| 5 | GPT-6 Astra | OpenAI | 82.9 |
| 6 | GPT-5.6 Sol | OpenAI | 80.5 |
| 7 | GPT-5.5 Pro | OpenAI | 78.0 |
| 8 | AI Co-Mathematician | 75.6 | |
| 9 | Claude Opus 5 | Anthropic | 73.2 |
| 10 | GPT-5.5 | OpenAI | 72.5 |
| 11 | GPT-5.6 Terra | OpenAI | 70.7 |
| 12 | GPT-5.6 Luna | OpenAI | 61.0 |
| 13 | GPT-5.4 Pro | OpenAI | 58.5 |
| 14 | Claude Opus 4.8 | Anthropic | 56.1 |
| 15 | GPT-5.4 | OpenAI | 49.0 |
| 16 | Qwen3.8 Max | Alibaba | 46.3 |
| 17 | GPT-5.2 Pro | OpenAI | 46.0 |
| 18 | Kimi K3 | Kimi | 39.0 |
| 19 | Gemini 3.7 Flash | 36.6 | |
| 20 | Qwen3.7-Max | Alibaba | 34.1 |
| 21 | Qwen3.8 Max (0902) | Alibaba | 34.1 |
| 22 | Claude Opus 4.7 | Anthropic | 31.7 |
| 23 | Grok 4.6 | xAI | 31.7 |
| 24 | GPT 5.2 | OpenAI | 31.7 |
| 25 | Claude Sonnet 5 | Anthropic | 29.3 |
| 26 | GLM-5.2 | Zhipu | 29.3 |
| 27 | GLM-5.3 | Zhipu | 29.3 |
| 28 | Claude Opus 4.6 | Anthropic | 26.8 |
| 29 | DeepSeek V4 Pro 0813 | DeepSeek | 26.8 |
| 30 | Gemini 3.1 Pro Preview | 26.8 | |
| 31 | Gemini 3.5 Flash | 26.8 | |
| 32 | Kimi K2.6 | Kimi | 25.6 |
| 33 | DeepSeek V4 Flash 0731 | DeepSeek | 24.4 |
| 34 | Grok 4.5 | xAI | 24.4 |
| 35 | Gemini 3.6 Flash | 21.9 | |
| 36 | Gemini 3.8 Flash | 21.9 | |
| 37 | GPT 5 | OpenAI | 21.9 |
| 38 | GPT-5 Pro | OpenAI | 19.5 |
| 39 | Gemini 3 Flash Preview | 17.1 | |
| 40 | GLM-5.3-Flash | Zhipu | 17.1 |
| 41 | Grok 4.20 | xAI | 17.1 |
| 42 | Inkling Small | Thinking Machines | 17.1 |
| 43 | Grok 4.3 Beta | xAI | 14.6 |
| 44 | GPT-5.4 nano | OpenAI | 12.2 |
| 45 | GPT 5 mini | OpenAI | 12.2 |
| 46 | Kimi K2.7 Code | Kimi | 12.2 |
| 47 | GPT-5.4 mini | OpenAI | 9.8 |
| 48 | Claude Opus 4.5 | Anthropic | 4.9 |
| 49 | Inkling | Thinking Machines | 4.9 |
| 50 | o4-mini | OpenAI | 4.9 |
| 51 | Claude Opus 4.1 | Anthropic | 2.4 |
| 52 | Claude Sonnet 4.5 | Anthropic | 2.4 |
| 53 | DeepSeek v4 | DeepSeek | 2.4 |
| 54 | GPT-5.5 Instant | OpenAI | 2.4 |
| 55 | GPT 5 nano | OpenAI | 2.4 |
| 56 | Gemini 2.5 Pro | 0.0 | |
| 57 | Gemini 3.5 Flash-Lite | 0.0 | |
| 58 | O3 Mini | OpenAI | 0.0 |