Puzzles and games have been central to AI development since the very beginning. Just as we humans like to test our smarts with crosswords or logic puzzles, developers can test how far models have advanced with a gaming gauntlet. The term "machine learning" was popularized in a 1959 article by the IBM computer scientist Arthur Samuel about an algorithm that learned to play checkers. Chess and the Chinese board game Go are famous AI test beds too.

Judged purely on its puzzling skills, AI is improving a lot—and quickly. In late 2024, a team of scientists from Columbia University showed that even the best models could figure out only 18% of the infamous New York Times Connections puzzles; by early 2025, some models could solve them near perfectly every time. But puzzles do more than just highlight the inexorable advance of AI capabilities.

Today's models still fumble: Subtle changes in classic riddles often trip them up, and visual puzzles are a particular weak spot. Here you'll have the chance to test your wits on puzzles that have stumped models at one time or another. Each one highlights at least one way in which machine and human cognition differ.

Spatial reasoning remains a domain where humans have a huge advantage. When a puzzle asks you to determine whether different images represent the same objects from different angles, though today's language models typically have the ability to analyze visual inputs, they still fail abysmally at these puzzles. For all the talk of how world models can help AI understand physical environments, LLMs still don't seem to be able to manipulate 3D objects the way spatial thinkers like architects and mechanical engineers can.