Chess Puzzles

Reasoning · questions rotate

Tactical chess positions with one correct continuation.

How far to trust it

A narrow skill, and not one most buyers need. It is in the composite because it discriminates well, not because it is representative.

Measured

Models scored

157

Spread between models

15.8

standard deviation, points

Measurement noise

3.8

published stderr

Weight

0.94

share of spread that is signal

The weight is (spread² − noise²) / spread². It is computed, never chosen. Most of this benchmark's spread is real, so it still separates models.

Every model scored

the median across configurations, so one heroic run cannot lead

Rank	Model	Lab	Score
1	GPT-6 Astra	OpenAI	72.0
2	GPT-5.5 Pro pre-release	OpenAI	64.0
3	GPT-5.6 Sol	OpenAI	64.0
4	Gemini 3.8 Flash	Google	61.0
5	GPT-5.4 Pro	OpenAI	58.6
6	Gemini 3.1 Pro Preview	Google	52.0
7	Claude Fable 5.1	Anthropic	47.0
8	DeepSeek V4 Pro 0813	DeepSeek	47.0
9	Gemini 3.7 Flash	Google	47.0
10	Gemini 3.5 Flash	Google	45.0
11	Claude Fable 5	Anthropic	41.0
12	GPT-5.6 Sol	OpenAI	41.0
13	Gemini 3.6 Flash	Google	40.0
14	GPT 5.2	OpenAI	40.0
15	GPT-5.5	OpenAI	40.0
16	Qwen3.8 Max (0902)	Alibaba	40.0
17	Gemini 3 Flash Preview	Google	39.0
18	GPT-5.4	OpenAI	38.0
19	GPT-5.6 Terra	OpenAI	38.0
20	Grok 4.5	xAI	36.0
21	Grok 4.6	xAI	35.5
22	Claude Opus 5	Anthropic	33.0
23	DeepSeek V4 Flash 0731	DeepSeek	33.0
24	o3	OpenAI	33.0
25	Claude Opus 4.8	Anthropic	31.5
26	Gemini 3 Pro Preview	Google	31.0
27	GPT-5.6 Luna	OpenAI	30.5
28	Qwen3.8 Max	Alibaba	29.0
29	Grok 4	xAI	28.0
30	GPT 5	OpenAI	26.5
31	Kimi K2.6	Kimi	26.0
32	Qwen3.6 35B-A3B	Alibaba	26.0
33	Claude Sonnet 5	Anthropic	25.5
34	Grok 4.3 Beta	xAI	25.0
35	Kimi K3	Kimi	25.0
36	Gemini 3.1 Flash-Lite	Google	24.0
37	Grok 4.20	xAI	24.0
38	Qwen3.7 Plus	Alibaba	24.0
39	GPT-5.4 nano	OpenAI	23.5
40	GPT 5.1	OpenAI	23.0
41	Qwen3.7 Flash	Alibaba	23.0
42	Qwen3.5 Plus	Alibaba	22.0
43	Qwen3.6 27B	Alibaba	22.0
44	Gemini 3.5 Flash-Lite	Google	21.0
45	GLM-5.3	Zhipu	21.0
46	GPT-5.4 mini	OpenAI	21.0
47	Inkling	Thinking Machines	21.0
48	Kimi K2.7 Code	Kimi	21.0
49	Qwen3.5 Flash	Alibaba	21.0
50	Claude Opus 4.7	Anthropic	20.0
51	Gemini 2.5 Pro	Google	20.0
52	gpt-oss-120b	OpenAI	20.0
53	Kimi K2 Instruct	Kimi	20.0
54	o4-mini	OpenAI	20.0
55	Qwen3.6 Max Preview	Alibaba	20.0
56	Qwen3.6 Flash	Alibaba	20.0
57	GPT 5 mini	OpenAI	19.8
58	GLM-5.1	Zhipu	19.0
59	Qwen3.7-Max	Alibaba	19.0
60	Inkling Small	Thinking Machines	18.0
61	GLM-5.2	Zhipu	17.5
62	GPT 5 nano	OpenAI	17.5
63	GPT-5.1	OpenAI	17.0
64	Qwen3.6 Plus	Alibaba	17.0
65	DeepSeek v4	DeepSeek	16.5
66	GLM-5.3-Flash	Zhipu	14.0
67	MiniMax M3	MiniMax	14.0
68	Claude Opus 4.6	Anthropic	13.5
69	Claude Opus 4.8	Anthropic	13.0
70	GPT 4o	OpenAI	13.0
71	Qwen3.5 397B-A17B	Alibaba	13.0
72	seed-oss-36b-instruct	—	13.0
73	Claude Opus 4.5	Anthropic	12.0
74	Claude Sonnet 4.5	Anthropic	12.0
75	GPT-5.5 Instant	OpenAI	12.0
76	Kimi K2.5	Kimi	12.0
77	Nemotron 3 Ultra	NVIDIA	12.0
78	Qwen3-235B-A22B	Alibaba	12.0
79	Qwen3.5 397B-A17B	Alibaba	12.0
80	Qwen3.5 9B	Alibaba	12.0
81	o1-2024-12-17	OpenAI	11.3
82	O3 Mini	OpenAI	10.7
83	GLM 5	Zhipu	10.0
84	GPT-5.5	OpenAI	10.0
85	Qwen3.6 27B	Alibaba	9.0
86	Qwen3.7 Flash	Alibaba	9.0
87	Qwen3.7 Plus	Alibaba	9.0
88	Claude Haiku 4.5	Anthropic	8.0
89	DeepSeek-V3.2-Exp	DeepSeek	7.5
90	Claude Opus 4.1	Anthropic	7.0
91	DeepSeek v4	DeepSeek	7.0
92	GPT 4.1 mini	OpenAI	7.0
93	GPT-5.6 Sol	OpenAI	7.0
94	Claude Sonnet 4.6	Anthropic	6.5
95	Gemma 4 26B-A4B IT	Google	6.0
96	GLM-4.7	Zhipu	6.0
97	GLM-5.2	Zhipu	6.0
98	GPT 4.1	OpenAI	6.0
99	GPT-4 Turbo	OpenAI	6.0
100	Qwen3.5-35B-A3B	Alibaba	5.5
101	Claude 3 Opus	Anthropic	5.0
102	Gemma 4 31B IT	Google	5.0
103	GPT-5.4	OpenAI	5.0
104	GPT-5.6 Terra	OpenAI	5.0
105	Qwen3-30B-A3B	Alibaba	5.0
106	Qwen3 8B	Alibaba	5.0
107	QwQ 32B	Alibaba	5.0
108	Claude Opus 4.5	Anthropic	4.0
109	Claude Sonnet 4.5	Anthropic	4.0
110	GPT-4	OpenAI	4.0
111	GPT-5.2	OpenAI	4.0
112	MiniMax M3	MiniMax	4.0
113	Qwen3 Max	Alibaba	4.0
114	qwen3-4b-instruct-2507	—	4.0
115	Qwen3.6 35B-A3B	Alibaba	4.0
116	DeepSeek-R1-0528	—	3.0
117	GPT-5.4 mini	OpenAI	3.0
118	GPT-5.4 nano	OpenAI	3.0
119	Magistral Small 1.2	Mistral	3.0
120	Qwen3 32B	Alibaba	3.0
121	Qwen3-30B-A3B	Alibaba	2.5
122	GPT-5.6 Luna	OpenAI	2.0
123	gpt-oss-20b	OpenAI	2.0
124	Qwen3-14B	Alibaba	2.0
125	Qwen3.5 9B	Alibaba	2.0
126	DeepSeek-R1-Distill-Qwen-14B	DeepSeek	1.0
127	DeepSeek-R1-Distill-Qwen-32B	DeepSeek	1.0
128	Mistral Small 3.1	Mistral	1.0
129	Mistral Small 3.2	Mistral	1.0
130	Phi-4	Microsoft	1.0
131	Qwen3-4B	Alibaba	1.0
132	DeepSeek LLM 67B	DeepSeek	0.0
133	DeepSeek-R1-Distill-Qwen-1.5B	DeepSeek	0.0
134	Gemma 3 12B	Google	0.0
135	Gemma 3 1B	Google	0.0
136	Gemma 3 27B	Google	0.0
137	Gemma 3 4B	Google	0.0
138	glm-4.7-flash	—	0.0
139	glm-4.7-flash_none	—	0.0
140	GPT-3.5 Turbo	OpenAI	0.0
141	GPT-4o mini	OpenAI	0.0
142	granite-4.0-1b	—	0.0
143	granite-4.0-350m	—	0.0
144	granite-4.0-micro	—	0.0
145	Llama 2-13B	Meta	0.0
146	Llama 2-7B	Meta	0.0
147	Llama 3.1-8B	Meta	0.0
148	Llama 3.2 1B	Meta	0.0
149	Llama 3-8B	Meta	0.0
150	Mistral 7B	Mistral	0.0
151	Mistral Small 3	Mistral	0.0
152	phi-3-mini 3.8B	Microsoft	0.0
153	Qwen2.5-32B	Alibaba	0.0
154	Qwen2.5-7B	Alibaba	0.0
155	Qwen3-1.7B	Alibaba	0.0
156	Qwen3.5-2B	Alibaba	0.0
157	Qwen3 8B	Alibaba	0.0

The benchmark's own page