FrontierMath tier 4 v2
Mathematics · fixed question set
The second version of the hardest FrontierMath tier.
How far to trust it
As tier 4. The v2 problems are newer, so contamination is lower still, and the two versions are not directly comparable.
Measured
Models scored
71
Spread between models
29.0
standard deviation, points
Measurement noise
6.5
published stderr
Weight
0.95
share of spread that is signal
The weight is (spread² − noise²) / spread². It is computed, never chosen. Most of this benchmark's spread is real, so it still separates models.
Every model scored
the median across configurations, so one heroic run cannot lead
| Rank | Model | Lab | Score |
|---|---|---|---|
| 1 | GPT-6.1 Sol | OpenAI | 100.0 |
| 2 | GPT-6 Astra | OpenAI | 97.6 |
| 3 | Claude Opus 5.5 | Anthropic | 95.0 |
| 4 | Claude Fable 5 | Anthropic | 90.2 |
| 5 | GPT-6 Sol | OpenAI | 90.0 |
| 6 | Claude Fable 5.1 | Anthropic | 87.8 |
| 7 | GPT-5.6 Sol | OpenAI | 82.9 |
| 8 | GPT-6 Astra | OpenAI | 82.9 |
| 9 | Claude Sonnet 5.5 | Anthropic | 80.5 |
| 10 | GPT-5.6 Sol | OpenAI | 80.5 |
| 11 | Gemini 4 Argon | 78.0 | |
| 12 | GPT-5.5 Pro | OpenAI | 78.0 |
| 13 | AI Co-Mathematician | 75.6 | |
| 14 | Claude Opus 5 | Anthropic | 73.2 |
| 15 | GPT-5.5 | OpenAI | 72.5 |
| 16 | GPT-5.6 Terra | OpenAI | 70.7 |
| 17 | Muse Wonder | Meta | 70.7 |
| 18 | GPT-5.6 Luna | OpenAI | 61.0 |
| 19 | GPT-5.4 Pro | OpenAI | 58.5 |
| 20 | Claude Opus 4.8 | Anthropic | 56.1 |
| 21 | GPT-6 Luna | OpenAI | 56.1 |
| 22 | GPT-5.4 | OpenAI | 49.0 |
| 23 | Claude Haiku 5.5 | Anthropic | 46.3 |
| 24 | Qwen3.8 Max | Alibaba | 46.3 |
| 25 | GPT-5.2 Pro | OpenAI | 46.0 |
| 26 | Muse Spark 1.3 | Meta | 43.9 |
| 27 | Kimi K3 | Kimi | 39.0 |
| 28 | Gemini 3.7 Flash | 36.6 | |
| 29 | Qwen3.7-Max | Alibaba | 34.1 |
| 30 | Qwen3.8 Max (0902) | Alibaba | 34.1 |
| 31 | Claude Opus 4.7 | Anthropic | 31.7 |
| 32 | Grok 4.6 | xAI | 31.7 |
| 33 | Muse Spark Open | Meta | 31.7 |
| 34 | GPT 5.2 | OpenAI | 31.7 |
| 35 | Claude Sonnet 5 | Anthropic | 29.3 |
| 36 | GLM-5.2 | Zhipu | 29.3 |
| 37 | GLM-5.3 | Zhipu | 29.3 |
| 38 | Claude Opus 4.6 | Anthropic | 26.8 |
| 39 | DeepSeek V4.1 Flash | DeepSeek | 26.8 |
| 40 | DeepSeek V4 Pro 0813 | DeepSeek | 26.8 |
| 41 | Gemini 3.1 Pro Preview | 26.8 | |
| 42 | Gemini 3.5 Flash | 26.8 | |
| 43 | Mistral Large 4 | Mistral | 26.8 |
| 44 | Kimi K2.6 | Kimi | 25.6 |
| 45 | DeepSeek V4 Flash 0731 | DeepSeek | 24.4 |
| 46 | Grok 4.5 | xAI | 24.4 |
| 47 | Gemini 3.6 Flash | 21.9 | |
| 48 | Gemini 3.8 Flash | 21.9 | |
| 49 | GPT 5 | OpenAI | 21.9 |
| 50 | GPT-5 Pro | OpenAI | 19.5 |
| 51 | Gemini 3 Flash Preview | 17.1 | |
| 52 | GLM-5.3-Flash | Zhipu | 17.1 |
| 53 | Grok 4.20 | xAI | 17.1 |
| 54 | Grok 4.7 | xAI | 17.1 |
| 55 | Inkling Small | Thinking Machines | 17.1 |
| 56 | Grok 4.3 Beta | xAI | 14.6 |
| 57 | GPT-5.4 nano | OpenAI | 12.2 |
| 58 | GPT 5 mini | OpenAI | 12.2 |
| 59 | Kimi K2.7 Code | Kimi | 12.2 |
| 60 | GPT-5.4 mini | OpenAI | 9.8 |
| 61 | Claude Opus 4.5 | Anthropic | 4.9 |
| 62 | Inkling | Thinking Machines | 4.9 |
| 63 | o4-mini | OpenAI | 4.9 |
| 64 | Claude Opus 4.1 | Anthropic | 2.4 |
| 65 | Claude Sonnet 4.5 | Anthropic | 2.4 |
| 66 | DeepSeek v4 | DeepSeek | 2.4 |
| 67 | GPT-5.5 Instant | OpenAI | 2.4 |
| 68 | GPT 5 nano | OpenAI | 2.4 |
| 69 | Gemini 2.5 Pro | 0.0 | |
| 70 | Gemini 3.5 Flash-Lite | 0.0 | |
| 71 | O3 Mini | OpenAI | 0.0 |