GPQA Diamond

Reasoning · fixed question set

448 graduate-level questions in biology, physics and chemistry, written so that a skilled non-expert with web access still scores near 34%.

How far to trust it

The most reported benchmark on this site, and the best understood. Epoch AI measured the gap between what labs report and what an independent run produces at about +0.7 points on average, inside their confidence intervals. On this benchmark a self-reported number is worth roughly what an independent one is. The set is fixed and public, so contamination rises over time.

Measured

Models scored

230

Spread between models

23.1

standard deviation, points

Measurement noise

2.5

published stderr

Weight

0.99

share of spread that is signal

The weight is (spread² − noise²) / spread². It is computed, never chosen. Most of this benchmark's spread is real, so it still separates models.

Every model scored

the median across configurations, so one heroic run cannot lead

Rank	Model	Lab	Score
1	Gemini 4 Argon	Google	96.4
2	GPT-6 Astra	OpenAI	95.8
3	Claude Sonnet 5.5	Anthropic	95.6
4	Gemini 3.8 Flash	Google	95.4
5	GPT-6.1 Sol	OpenAI	95.4
6	Gemini 3.7 Flash	Google	94.8
7	GPT-5.4 Pro	OpenAI	94.6
8	GPT-6 Sol	OpenAI	94.3
9	Gemini 3.1 Pro Preview	Google	94.3
10	GPT-5.5 Pro pre-release	OpenAI	93.9
11	Grok 4.6	xAI	93.6
12	Grok 4.5	xAI	93.4
13	Claude Opus 5	Anthropic	92.9
14	Grok 4.7	xAI	92.7
15	Qwen3.8 Max	Alibaba	92.7
16	Gemini 3 Pro Preview	Google	92.6
17	GPT-5.5	OpenAI	92.3
18	Qwen3.8 Max (0902)	Alibaba	92.3
19	Kimi K3	Kimi	91.9
20	GPT-5.6 Sol	OpenAI	91.7
21	DeepSeek V4 Pro 0813	DeepSeek	91.7
22	DeepSeek V4 Flash 0731	DeepSeek	91.0
23	GLM-5.3	Zhipu	90.9
24	MiniMax M3	MiniMax	90.9
25	Qwen3.7-Max	Alibaba	90.9
26	Kimi K2.6	Kimi	90.8
27	Claude Opus 5.5	Anthropic	90.6
28	GPT-6 Luna	OpenAI	90.5
29	GPT-5.6 Terra	OpenAI	90.3
30	DeepSeek v4	DeepSeek	90.3
31	GLM-5.3-Flash	Zhipu	90.2
32	GLM-5.1	Zhipu	89.9
33	GLM-5.2	Zhipu	89.9
34	DeepSeek V4.1 Flash	DeepSeek	89.8
35	Muse Spark	Meta	89.8
36	Claude Opus 4.8	Anthropic	89.7
37	Claude Haiku 5.5	Anthropic	89.7
38	GPT-5.4	OpenAI	89.4
39	Grok 4.20	xAI	89.3
40	Gemini 3.5 Flash	Google	88.9
41	Grok 4.3 Beta	xAI	88.8
42	Claude Opus 4.6	Anthropic	88.8
43	Muse Spark Open	Meta	88.6
44	Inkling Small	Thinking Machines	88.5
45	Qwen3.6 Plus	Alibaba	88.4
46	Inkling	Thinking Machines	88.3
47	Claude Opus 4.7	Anthropic	88.3
48	Mistral Large 4	Mistral	88.1
49	GPT 5.2	OpenAI	88.0
50	Kimi K2.7 Code	Kimi	87.9
51	Qwen3.7 Plus	Alibaba	87.9
52	GLM 5	Zhipu	87.8
53	Kimi K2.5	Kimi	87.6
54	Qwen3.6 Max Preview	Alibaba	87.4
55	Grok 4	xAI	87.0
56	GPT-5.6 Luna	OpenAI	87.0
57	Gemini 3.6 Flash	Google	86.4
58	Qwen3.5 397B-A17B	Alibaba	86.4
59	GPT 5.1	OpenAI	86.3
60	Gemini 3 Flash Preview	Google	86.3
61	Qwen3.5 397B-A17B	Alibaba	85.9
62	Qwen3.6 27B	Alibaba	85.9
63	Claude Opus 4.5	Anthropic	85.8
64	Claude Sonnet 5	Anthropic	85.4
65	Claude Opus 4.8	Anthropic	85.3
66	GPT 5	OpenAI	85.3
67	Nemotron 3 Ultra	NVIDIA	85.3
68	Gemini 2.5 Pro	Google	85.3
69	GPT-5.4 mini	OpenAI	85.2
70	Qwen3.5 Plus	Alibaba	84.8
71	Qwen3.6 27B	Alibaba	84.8
72	Qwen3.6 35B-A3B	Alibaba	84.8
73	Kimi K2 Instruct	Kimi	84.2
74	Gemini 2.5 Pro Exp	Google	83.8
75	Qwen3.6 35B-A3B	Alibaba	83.8
76	Claude Fable 5	Anthropic	83.3
77	Claude Sonnet 4.6	Anthropic	83.3
78	GLM-4.7	Zhipu	83.3
79	Qwen3.6 Flash	Alibaba	83.3
80	GPT-5.6 Sol	OpenAI	82.8
81	GPT-5.5 Instant	OpenAI	82.5
82	Qwen3.5-35B-A3B	Alibaba	82.3
83	Qwen3.5 Flash	Alibaba	82.3
84	Qwen3.7 Flash	Alibaba	82.3
85	Qwen3.7 Plus	Alibaba	81.8
86	Claude Sonnet 4.5	Anthropic	81.7
87	MiniMax M3	MiniMax	81.3
88	o3	OpenAI	80.8
89	Qwen3.7 Flash	Alibaba	80.8
90	Claude Opus 4.5	Anthropic	80.7
91	Qwen3-235B-A22B	Alibaba	80.0
92	Qwen3.5 9B	Alibaba	79.0
93	Qwen3.5 9B	Alibaba	78.9
94	o4-mini	OpenAI	77.5
95	DeepSeek-V3.2-Exp	DeepSeek	77.3
96	GPT-5.5	OpenAI	77.3
97	GPT-5.6 Terra	OpenAI	77.3
98	Claude Sonnet 4	Anthropic	76.8
99	Claude 3.7 Sonnet	Anthropic	76.8
100	Claude Opus 4.1	Anthropic	76.8
101	DeepSeek-R1	DeepSeek	76.3
102	Gemini 3.5 Flash-Lite	Google	75.8
103	Gemma 4 31B IT	Google	75.8
104	gpt-oss-120b	OpenAI	75.8
105	Gemini 2.5 Pro Preview	Google	75.8
106	Grok-3 mini	xAI	75.4
107	GPT-5.4 nano	OpenAI	75.3
108	GPT-5.4	OpenAI	74.8
109	Gemini 3.1 Flash-Lite	Google	74.2
110	Claude Sonnet 4.5	Anthropic	73.7
111	GPT 5 mini	OpenAI	73.3
112	DeepSeek v4	DeepSeek	73.2
113	Gemma 4 26B-A4B IT	Google	73.2
114	GPT-5.2	OpenAI	73.2
115	O3 Mini	OpenAI	73.2
116	Claude Opus 4	Anthropic	72.7
117	Qwen3 Max	Alibaba	72.6
118	seed-oss-36b-instruct	—	71.5
119	GLM-5.2	Zhipu	71.2
120	Qwen3	Alibaba	70.7
121	o1-2024-12-17	OpenAI	69.3
122	DeepSeek R1	DeepSeek	69.2
123	GPT-4.5	OpenAI	68.7
124	DeepSeek-V3	DeepSeek	67.6
125	Grok 3 Beta	xAI	67.6
126	GPT 5 nano	OpenAI	67.4
127	Llama 4 Maverick Instruct	Meta	67.0
128	GPT 4.1	OpenAI	66.9
129	GPT-5.1	OpenAI	66.7
130	GPT 4.1 mini	OpenAI	65.8
131	Claude Haiku 4.5	Anthropic	65.8
132	Gemini 2.0 Pro	Google	65.7
133	QwQ-Plus	Alibaba	65.4
134	QwQ 32B	Alibaba	65.3
135	DeepSeek-R1-Distill-Qwen-32B	DeepSeek	64.1
136	GPT-5.4 mini	OpenAI	64.1
137	GPT-5.6 Luna	OpenAI	63.6
138	Qwen3-30B-A3B	Alibaba	62.8
139	o1-mini	OpenAI	60.9
140	Gemini 2.0 Flash Thinking	Google	60.6
141	glm-4.7-flash	—	60.5
142	Qwen3 32B	Alibaba	59.9
143	Mistral Medium 3	Mistral	59.5
144	Qwen3-14B	Alibaba	58.6
145	Qwen3 8B	Alibaba	56.8
146	DeepSeek-V3	DeepSeek	56.5
147	Qwen2.5-Max	Alibaba	56.1
148	Magistral Small 1.0	Mistral	56.1
149	Phi-4	Microsoft	56.1
150	Qwen3-30B-A3B	Alibaba	56.0
151	DeepSeek-R1-Distill-Llama-70B	DeepSeek	55.7
152	GPT-5.4 nano	OpenAI	55.6
153	Claude 3.5 Sonnet	Anthropic	55.3
154	Claude 3.5 Sonnet	Anthropic	54.0
155	Grok-2	xAI	53.8
156	gpt-oss-20b	OpenAI	53.2
157	Llama 4 Scout Instruct	Meta	51.8
158	Gemini 1.5 Pro	Google	51.5
159	Llama 3.1-405B	Meta	50.9
160	Mistral Large 2	Mistral	50.2
161	Qwen2.5-72B	Alibaba	49.1
162	Mistral Small 3.2	Mistral	49.0
163	GPT-4.1 nano	OpenAI	48.9
164	GPT 4o	OpenAI	48.7
165	Qwen Plus	Alibaba	48.1
166	Qwen3-4B	Alibaba	48.0
167	Gemma 3 27B	Google	47.7
168	Magistral Small 1.2	Mistral	47.6
169	Qwen3 8B	Alibaba	47.5
170	Llama 3.3 70B	Meta	47.4
171	Claude 3 Opus	Anthropic	47.2
172	GPT-4 Turbo	OpenAI	46.6
173	Mistral Small 3	Mistral	46.3
174	Tülu 3 70B	Ai2	46.3
175	Qwen2.5-32B	Alibaba	46.1
176	qwen3-4b-instruct-2507	—	45.8
177	glm-4.7-flash_none	—	45.1
178	DeepSeek-R1-Distill-Qwen-14B	DeepSeek	44.7
179	Mistral Small 3.1	Mistral	44.7
180	Llama 3.1-70B	Meta	44.2
181	Gemini 1.5 Flash	Google	43.8
182	WizardLM-2 8x22B	Microsoft	43.4
183	GPT-4 Turbo	OpenAI	42.3
184	Qwen Turbo	Alibaba	41.8
185	Llama 3.2 90B	Meta	41.0
186	Qwen2-72B	Alibaba	40.8
187	Claude 3 Sonnet	Anthropic	40.6
188	Llama 3-70B	Meta	40.6
189	Gemma 3 12B	Google	39.5
190	Mistral Large	Mistral	38.8
191	Claude 3.5 Haiku	Anthropic	38.1
192	GPT-4o mini	OpenAI	37.7
193	Hermes 2 Theta Llama-3 70B	Nous Research	37.5
194	Gemma 2 27B	Google	36.5
195	Claude 3 Haiku	Anthropic	36.3
196	GPT-4	OpenAI	35.7
197	Qwen2.5-7B	Alibaba	35.5
198	Claude 2	Anthropic	34.7
199	Qwen3-1.7B	Alibaba	34.3
200	Mixtral 8x22B	Mistral	34.1
201	Gemini 1.0 Pro	Google	34.0
202	Eurus-2-7B-PRIME	Tsinghua University	33.9
203	DeepSeek-R1-Distill-Qwen-1.5B	DeepSeek	33.6
204	Claude 2.1	Anthropic	33.0
205	Gemini 1.5 Flash 8B	Google	33.0
206	DBRX	Databricks	32.9
207	Yi-1.5-34B	01.AI	32.0
208	Qwen1.5-32B	Alibaba	30.7
209	GPT-4	OpenAI	30.6
210	Mixtral 8x7B	Mistral	30.2
211	Mistral NeMo	Mistral	29.9
212	Qwen1.5-72B	Alibaba	28.8
213	granite-4.0-micro	—	28.3
214	GPT-3.5 Turbo	OpenAI	27.6
215	phi-3-medium 14B	Microsoft	27.6
216	Gemma 2 9B	Google	27.5
217	Ministral 8B	Mistral	27.1
218	Llama 3.1-8B	Meta	27.0
219	Llama 2-70B	Meta	26.3
220	Llama 3-8B	Meta	26.1
221	Ministral 3B	Mistral	25.2
222	DeepSeek LLM 67B	DeepSeek	24.6
223	granite-4.0-1b	—	24.0
224	Llama 3.2 1B	Meta	23.9
225	Gemma 3 4B	Google	23.2
226	Gemma 3 1B	Google	19.9
227	Yi-34B	01.AI	14.7
228	Mistral 7B	Mistral	14.2
229	granite-4.0-350m	—	11.2
230	DeepSeek-R1-0528	—	9.3

The benchmark's own page