FrontierMath v1
Mathematics · fixed question set
Research-level mathematics problems, most written by working mathematicians for this benchmark.
How far to trust it
Almost all problems are held private, so contamination is very low. The trade-off is that you cannot inspect what was asked. Ten problems are public; every figure here is from the private set.
Measured
Models scored
81
Spread between models
23.5
standard deviation, points
Measurement noise
2.6
published stderr
Weight
0.99
share of spread that is signal
The weight is (spread² − noise²) / spread². It is computed, never chosen. Most of this benchmark's spread is real, so it still separates models.
Every model scored
the median across configurations, so one heroic run cannot lead
| Rank | Model | Lab | Score |
|---|---|---|---|
| 1 | Claude Fable 5 | Anthropic | 100.0 |
| 2 | Claude Opus 4.7 | Anthropic | 66.9 |
| 3 | Claude Opus 4.6 | Anthropic | 64.8 |
| 4 | Kimi K2.6 | Kimi | 64.5 |
| 5 | GPT-5.4 | OpenAI | 63.8 |
| 6 | Gemini 3.1 Pro Preview | 62.9 | |
| 7 | GPT-5 Pro | OpenAI | 60.0 |
| 8 | Gemini 3.5 Flash | 59.5 | |
| 9 | Gemini 3 Pro Preview | 58.8 | |
| 10 | GLM-5.1 | Zhipu | 56.7 |
| 11 | Claude Sonnet 4.6 | Anthropic | 56.2 |
| 12 | GPT-5.5 | OpenAI | 51.7 |
| 13 | GPT-5.5 Pro pre-release | OpenAI | 51.7 |
| 14 | GPT-5.4 Pro | OpenAI | 50.0 |
| 15 | Gemini 3 Flash Preview | 47.8 | |
| 16 | GPT 5 | OpenAI | 47.4 |
| 17 | Claude Opus 4.8 | Anthropic | 47.2 |
| 18 | GPT 5.2 | OpenAI | 44.6 |
| 19 | GPT-5.4 nano | OpenAI | 42.9 |
| 20 | GPT-5.4 mini | OpenAI | 39.1 |
| 21 | Muse Spark | Meta | 39.0 |
| 22 | Claude Opus 4.6 | Anthropic | 38.3 |
| 23 | Qwen3.6 Plus | Alibaba | 38.1 |
| 24 | Qwen3.6 Max Preview | Alibaba | 36.5 |
| 25 | Qwen3.5 Plus | Alibaba | 35.5 |
| 26 | GPT 5 mini | OpenAI | 29.4 |
| 27 | Gemini 2.5 Deep Think | 29.0 | |
| 28 | Kimi K2.5 | Kimi | 27.9 |
| 29 | Kimi K2 Instruct | Kimi | 27.1 |
| 30 | Gemini 2.5 Pro | 27.1 | |
| 31 | GPT 5.1 | OpenAI | 26.9 |
| 32 | Claude Opus 4.5 | Anthropic | 25.4 |
| 33 | o4-mini | OpenAI | 22.4 |
| 34 | DeepSeek-V3.2-Exp | DeepSeek | 22.1 |
| 35 | Claude Opus 4.5 | Anthropic | 20.3 |
| 36 | Gemini 2.5 Pro Preview | 20.2 | |
| 37 | Grok 4 | xAI | 19.7 |
| 38 | O3 Mini | OpenAI | 17.9 |
| 39 | Qwen3.6 Flash | Alibaba | 15.2 |
| 40 | Claude Sonnet 4.5 | Anthropic | 14.4 |
| 41 | Qwen3-235B-A22B | Alibaba | 14.2 |
| 42 | GPT 5 nano | OpenAI | 11.4 |
| 43 | o3 | OpenAI | 10.9 |
| 44 | GLM 5 | Zhipu | 9.3 |
| 45 | Qwen3.5 Flash | Alibaba | 8.1 |
| 46 | GPT 4.1 mini | OpenAI | 7.2 |
| 47 | Gemini 2.5 Flash | 4.8 | |
| 48 | Claude Sonnet 4.5 | Anthropic | 4.7 |
| 49 | o1-2024-12-17 | OpenAI | 4.7 |
| 50 | Claude Haiku 4.5 | Anthropic | 3.5 |
| 51 | Claude Opus 4 | Anthropic | 3.2 |
| 52 | Claude Opus 4.1 | Anthropic | 2.9 |
| 53 | GPT 4.1 | OpenAI | 2.8 |
| 54 | GLM-4.7 | Zhipu | 2.4 |
| 55 | Grok-3 mini | xAI | 2.2 |
| 56 | GPT-5.1 | OpenAI | 2.1 |
| 57 | Claude Sonnet 4 | Anthropic | 2.1 |
| 58 | GLM 4.6 | Zhipu | 1.9 |
| 59 | Grok 3 Beta | xAI | 1.9 |
| 60 | Claude 3.7 Sonnet | Anthropic | 1.6 |
| 61 | Claude 3.5 Sonnet | Anthropic | 1.0 |
| 62 | DeepSeek-V3 | DeepSeek | 0.9 |
| 63 | Gemini 2.0 Flash Thinking | 0.9 | |
| 64 | Qwen Plus | Alibaba | 0.9 |
| 65 | o1-mini | OpenAI | 0.8 |
| 66 | Claude 3.5 Sonnet | Anthropic | 0.5 |
| 67 | GPT-4.1 nano | OpenAI | 0.5 |
| 68 | Qwen2.5-Max | Alibaba | 0.5 |
| 69 | Grok-2 | xAI | 0.3 |
| 70 | Llama 4 Maverick Instruct | Meta | 0.3 |
| 71 | Mistral Medium 3 | Mistral | 0.2 |
| 72 | Claude 3.5 Haiku | Anthropic | 0.2 |
| 73 | GPT 4o | OpenAI | 0.2 |
| 74 | Mistral Large 2 | Mistral | 0.2 |
| 75 | DeepSeek-V3 | DeepSeek | 0.0 |
| 76 | Gemini 1.5 Flash | 0.0 | |
| 77 | Gemini 2.0 Pro | 0.0 | |
| 78 | GLM 4.5 | Zhipu | 0.0 |
| 79 | Llama 4 Scout Instruct | Meta | 0.0 |
| 80 | Magistral Small 1.0 | Mistral | 0.0 |
| 81 | Qwen3 | Alibaba | 0.0 |