A student neural network learned to recognize handwritten digits without ever seeing a single image or label. Its entire training diet consisted of random static, and its only task was to copy a few extra numbers produced by a teacher network. When tested on real MNIST digits, the student achieved over 50% accuracy, reaching up to 93.8% in some trials.
This phenomenon, called subliminal learning, was demonstrated by researchers at Anthropic, UC Berkeley, and other institutions. The author reproduced the experiment using plain NumPy, showing it runs in about 20 minutes on a free Colab CPU. The key condition for success is that both teacher and student must start from the same random weights.
If the student begins with different weights, no knowledge transfers. The student’s loss function never directly trains the digit outputs—those weights remain unchanged. Instead, learning occurs in the shared layers beneath, triggered by copying auxiliary logits.
The effect emerges quickly: after just 10% of training steps, accuracy reached 43.4%, crossing 50% by step 468.
Source: Towards Data Science · Summarized by HeadlinesBriefing