In 2022, researchers trained a tiny neural network on modular addition, a simple clock-math task. The model quickly memorized training data but failed on unseen examples. After thousands of additional training steps with no visible changes, its performance on new problems jumped from near-random to nearly perfect.

This phenomenon, named grokking, shows that a model can appear finished yet later develop true understanding. The network essentially rediscovered trigonometry, representing numbers as points on a circle and using rotation to solve problems. This delayed insight challenges assumptions about when learning truly occurs.