GPQA Diamond

Reasoning · fixed question set

448 graduate-level questions in biology, physics and chemistry, written so that a skilled non-expert with web access still scores near 34%.

How far to trust it

The most reported benchmark on this site, and the best understood. Epoch AI measured the gap between what labs report and what an independent run produces at about +0.7 points on average, inside their confidence intervals. On this benchmark a self-reported number is worth roughly what an independent one is. The set is fixed and public, so contamination rises over time.

Measured

Models scored

219

Spread between models

22.8

standard deviation, points

Measurement noise

2.5

published stderr

Weight

0.99

share of spread that is signal

The weight is (spread² − noise²) / spread². It is computed, never chosen. Most of this benchmark's spread is real, so it still separates models.

Every model scored

the median across configurations, so one heroic run cannot lead

Rank	Model	Lab	Score
1	GPT-6 Astra	OpenAI	95.8
2	Gemini 3.8 Flash	Google	95.4
3	Gemini 3.7 Flash	Google	94.8
4	GPT-5.4 Pro	OpenAI	94.6
5	Gemini 3.1 Pro Preview	Google	94.3
6	GPT-5.5 Pro pre-release	OpenAI	93.9
7	Grok 4.6	xAI	93.6
8	Grok 4.5	xAI	93.4
9	Claude Opus 5	Anthropic	92.9
10	Qwen3.8 Max	Alibaba	92.7
11	Gemini 3 Pro Preview	Google	92.6
12	GPT-5.5	OpenAI	92.3
13	Qwen3.8 Max (0902)	Alibaba	92.3
14	Kimi K3	Kimi	91.9
15	GPT-5.6 Sol	OpenAI	91.7
16	DeepSeek V4 Pro 0813	DeepSeek	91.7
17	DeepSeek V4 Flash 0731	DeepSeek	91.0
18	GLM-5.3	Zhipu	90.9
19	MiniMax M3	MiniMax	90.9
20	Qwen3.7-Max	Alibaba	90.9
21	Kimi K2.6	Kimi	90.8
22	GPT-5.6 Terra	OpenAI	90.3
23	DeepSeek v4	DeepSeek	90.3
24	GLM-5.3-Flash	Zhipu	90.2
25	GLM-5.1	Zhipu	89.9
26	GLM-5.2	Zhipu	89.9
27	Muse Spark	Meta	89.8
28	Claude Opus 4.8	Anthropic	89.7
29	GPT-5.4	OpenAI	89.4
30	Grok 4.20	xAI	89.3
31	Gemini 3.5 Flash	Google	88.9
32	Grok 4.3 Beta	xAI	88.8
33	Claude Opus 4.6	Anthropic	88.8
34	Inkling Small	Thinking Machines	88.5
35	Qwen3.6 Plus	Alibaba	88.4
36	Inkling	Thinking Machines	88.3
37	Claude Opus 4.7	Anthropic	88.3
38	GPT 5.2	OpenAI	88.0
39	Kimi K2.7 Code	Kimi	87.9
40	Qwen3.7 Plus	Alibaba	87.9
41	GLM 5	Zhipu	87.8
42	Kimi K2.5	Kimi	87.6
43	Qwen3.6 Max Preview	Alibaba	87.4
44	Grok 4	xAI	87.0
45	GPT-5.6 Luna	OpenAI	87.0
46	Gemini 3.6 Flash	Google	86.4
47	Qwen3.5 397B-A17B	Alibaba	86.4
48	GPT 5.1	OpenAI	86.3
49	Gemini 3 Flash Preview	Google	86.3
50	Qwen3.5 397B-A17B	Alibaba	85.9
51	Qwen3.6 27B	Alibaba	85.9
52	Claude Opus 4.5	Anthropic	85.8
53	Claude Sonnet 5	Anthropic	85.4
54	Claude Opus 4.8	Anthropic	85.3
55	GPT 5	OpenAI	85.3
56	Nemotron 3 Ultra	NVIDIA	85.3
57	Gemini 2.5 Pro	Google	85.3
58	GPT-5.4 mini	OpenAI	85.2
59	Qwen3.5 Plus	Alibaba	84.8
60	Qwen3.6 27B	Alibaba	84.8
61	Qwen3.6 35B-A3B	Alibaba	84.8
62	Kimi K2 Instruct	Kimi	84.2
63	Gemini 2.5 Pro Exp	Google	83.8
64	Qwen3.6 35B-A3B	Alibaba	83.8
65	Claude Fable 5	Anthropic	83.3
66	Claude Sonnet 4.6	Anthropic	83.3
67	GLM-4.7	Zhipu	83.3
68	Qwen3.6 Flash	Alibaba	83.3
69	GPT-5.6 Sol	OpenAI	82.8
70	GPT-5.5 Instant	OpenAI	82.5
71	Qwen3.5-35B-A3B	Alibaba	82.3
72	Qwen3.5 Flash	Alibaba	82.3
73	Qwen3.7 Flash	Alibaba	82.3
74	Qwen3.7 Plus	Alibaba	81.8
75	Claude Sonnet 4.5	Anthropic	81.7
76	MiniMax M3	MiniMax	81.3
77	o3	OpenAI	80.8
78	Qwen3.7 Flash	Alibaba	80.8
79	Claude Opus 4.5	Anthropic	80.7
80	Qwen3-235B-A22B	Alibaba	80.0
81	Qwen3.5 9B	Alibaba	79.0
82	Qwen3.5 9B	Alibaba	78.9
83	o4-mini	OpenAI	77.5
84	DeepSeek-V3.2-Exp	DeepSeek	77.3
85	GPT-5.5	OpenAI	77.3
86	GPT-5.6 Terra	OpenAI	77.3
87	Claude Sonnet 4	Anthropic	76.8
88	Claude 3.7 Sonnet	Anthropic	76.8
89	Claude Opus 4.1	Anthropic	76.8
90	DeepSeek-R1	DeepSeek	76.3
91	Gemini 3.5 Flash-Lite	Google	75.8
92	Gemma 4 31B IT	Google	75.8
93	gpt-oss-120b	OpenAI	75.8
94	Gemini 2.5 Pro Preview	Google	75.8
95	Grok-3 mini	xAI	75.4
96	GPT-5.4 nano	OpenAI	75.3
97	GPT-5.4	OpenAI	74.8
98	Gemini 3.1 Flash-Lite	Google	74.2
99	Claude Sonnet 4.5	Anthropic	73.7
100	GPT 5 mini	OpenAI	73.3
101	DeepSeek v4	DeepSeek	73.2
102	Gemma 4 26B-A4B IT	Google	73.2
103	GPT-5.2	OpenAI	73.2
104	O3 Mini	OpenAI	73.2
105	Claude Opus 4	Anthropic	72.7
106	Qwen3 Max	Alibaba	72.6
107	seed-oss-36b-instruct	—	71.5
108	GLM-5.2	Zhipu	71.2
109	Qwen3	Alibaba	70.7
110	o1-2024-12-17	OpenAI	69.3
111	DeepSeek R1	DeepSeek	69.2
112	GPT-4.5	OpenAI	68.7
113	DeepSeek-V3	DeepSeek	67.6
114	Grok 3 Beta	xAI	67.6
115	GPT 5 nano	OpenAI	67.4
116	Llama 4 Maverick Instruct	Meta	67.0
117	GPT 4.1	OpenAI	66.9
118	GPT-5.1	OpenAI	66.7
119	GPT 4.1 mini	OpenAI	65.8
120	Claude Haiku 4.5	Anthropic	65.8
121	Gemini 2.0 Pro	Google	65.7
122	QwQ-Plus	Alibaba	65.4
123	QwQ 32B	Alibaba	65.3
124	DeepSeek-R1-Distill-Qwen-32B	DeepSeek	64.1
125	GPT-5.4 mini	OpenAI	64.1
126	GPT-5.6 Luna	OpenAI	63.6
127	Qwen3-30B-A3B	Alibaba	62.8
128	o1-mini	OpenAI	60.9
129	Gemini 2.0 Flash Thinking	Google	60.6
130	glm-4.7-flash	—	60.5
131	Qwen3 32B	Alibaba	59.9
132	Mistral Medium 3	Mistral	59.5
133	Qwen3-14B	Alibaba	58.6
134	Qwen3 8B	Alibaba	56.8
135	DeepSeek-V3	DeepSeek	56.5
136	Qwen2.5-Max	Alibaba	56.1
137	Magistral Small 1.0	Mistral	56.1
138	Phi-4	Microsoft	56.1
139	Qwen3-30B-A3B	Alibaba	56.0
140	DeepSeek-R1-Distill-Llama-70B	DeepSeek	55.7
141	GPT-5.4 nano	OpenAI	55.6
142	Claude 3.5 Sonnet	Anthropic	55.3
143	Claude 3.5 Sonnet	Anthropic	54.0
144	Grok-2	xAI	53.8
145	gpt-oss-20b	OpenAI	53.2
146	Llama 4 Scout Instruct	Meta	51.8
147	Gemini 1.5 Pro	Google	51.5
148	Llama 3.1-405B	Meta	50.9
149	Mistral Large 2	Mistral	50.2
150	Qwen2.5-72B	Alibaba	49.1
151	Mistral Small 3.2	Mistral	49.0
152	GPT-4.1 nano	OpenAI	48.9
153	GPT 4o	OpenAI	48.7
154	Qwen Plus	Alibaba	48.1
155	Qwen3-4B	Alibaba	48.0
156	Gemma 3 27B	Google	47.7
157	Magistral Small 1.2	Mistral	47.6
158	Qwen3 8B	Alibaba	47.5
159	Llama 3.3 70B	Meta	47.4
160	Claude 3 Opus	Anthropic	47.2
161	GPT-4 Turbo	OpenAI	46.6
162	Mistral Small 3	Mistral	46.3
163	Tülu 3 70B	Ai2	46.3
164	Qwen2.5-32B	Alibaba	46.1
165	qwen3-4b-instruct-2507	—	45.8
166	glm-4.7-flash_none	—	45.1
167	DeepSeek-R1-Distill-Qwen-14B	DeepSeek	44.7
168	Mistral Small 3.1	Mistral	44.7
169	Llama 3.1-70B	Meta	44.2
170	Gemini 1.5 Flash	Google	43.8
171	WizardLM-2 8x22B	Microsoft	43.4
172	GPT-4 Turbo	OpenAI	42.3
173	Qwen Turbo	Alibaba	41.8
174	Llama 3.2 90B	Meta	41.0
175	Qwen2-72B	Alibaba	40.8
176	Claude 3 Sonnet	Anthropic	40.6
177	Llama 3-70B	Meta	40.6
178	Gemma 3 12B	Google	39.5
179	Mistral Large	Mistral	38.8
180	Claude 3.5 Haiku	Anthropic	38.1
181	GPT-4o mini	OpenAI	37.7
182	Hermes 2 Theta Llama-3 70B	Nous Research	37.5
183	Gemma 2 27B	Google	36.5
184	Claude 3 Haiku	Anthropic	36.3
185	GPT-4	OpenAI	35.7
186	Qwen2.5-7B	Alibaba	35.5
187	Claude 2	Anthropic	34.7
188	Qwen3-1.7B	Alibaba	34.3
189	Mixtral 8x22B	Mistral	34.1
190	Gemini 1.0 Pro	Google	34.0
191	Eurus-2-7B-PRIME	Tsinghua University	33.9
192	DeepSeek-R1-Distill-Qwen-1.5B	DeepSeek	33.6
193	Claude 2.1	Anthropic	33.0
194	Gemini 1.5 Flash 8B	Google	33.0
195	DBRX	Databricks	32.9
196	Yi-1.5-34B	01.AI	32.0
197	Qwen1.5-32B	Alibaba	30.7
198	GPT-4	OpenAI	30.6
199	Mixtral 8x7B	Mistral	30.2
200	Mistral NeMo	Mistral	29.9
201	Qwen1.5-72B	Alibaba	28.8
202	granite-4.0-micro	—	28.3
203	GPT-3.5 Turbo	OpenAI	27.6
204	phi-3-medium 14B	Microsoft	27.6
205	Gemma 2 9B	Google	27.5
206	Ministral 8B	Mistral	27.1
207	Llama 3.1-8B	Meta	27.0
208	Llama 2-70B	Meta	26.3
209	Llama 3-8B	Meta	26.1
210	Ministral 3B	Mistral	25.2
211	DeepSeek LLM 67B	DeepSeek	24.6
212	granite-4.0-1b	—	24.0
213	Llama 3.2 1B	Meta	23.9
214	Gemma 3 4B	Google	23.2
215	Gemma 3 1B	Google	19.9
216	Yi-34B	01.AI	14.7
217	Mistral 7B	Mistral	14.2
218	granite-4.0-350m	—	11.2
219	DeepSeek-R1-0528	—	9.3

The benchmark's own page