HeadlinesBriefing favicon HeadlinesBriefing.com

Language Models Under Pedagogically‑Controlled Knowledge Exposure

Hacker News •
×

The project trains language models on a strictly defined curriculum. The 88B‑token corpus, distilled from Fine Web‑Edu and filtered to align with Common Core K–5 standards, removes any concepts above Grade 5. Models are trained from scratch on this data, producing versions at 0.6B, 1.3B, and 5B parameters.

Three scales of the Little Learner are released, each paired with a matched Psychiatry‑unfiltered control that shares architecture, tokens, and recipe. The controls allow a clean comparison of what the curriculum teaches versus what an unfiltered model learns.

Findings show that scaling, SFT+GRPO post‑training, and in‑context learning only amplify what the curriculum already provides. None of these interventions meaningfully improves out‑of‑scope performance, indicating that the pretraining filter sets the effective capability ceiling.

Future work explores reinforcement learning to create new capabilities, continual learning to observe concept acquisition, and educational science experiments comparing model and child learners at the same exposure boundary.