Trip Venturella dropped Mr. Chatterbox, a surprisingly small language model trained exclusively on British Library texts spanning 1837 to 1899. With only 340 million parameters, comparable to GPT-2-Medium, the goal was testing the conversational viability of models built solely on out-of-copyright material. This project addresses the growing frustration over reliance on massive, often unlicensed, scraped datasets for modern LLM development.

Factoring in the 2.93 billion tokens used for training, the resulting model—a mere 2.05GB on disk—performs poorly, resembling a sophisticated Markov chain more than a useful LLM. Chinchilla scaling laws suggest this model needed significantly more data to achieve coherence, perhaps four times the current corpus size. Still, the ethical purity makes it an interesting artifact for developers.

Developers can now run this quaint creation locally using the `llm` framework, thanks to a new plugin, `llm-mrchatterbox`. The process involved training via Andrej Karpathy’s nanochat and then leveraging Claude Code to scaffold the necessary integration script. Running the model requires fetching the weights via HuggingFace Spaces upon the first prompt execution.

This exercise proves that while ethically sourced, historical data alone struggles to build modern conversational agents, the process itself offers a valuable proof-of-concept. The ability to deploy and manage this model via the LLM framework demonstrates streamlined tooling for custom, small-scale deployments.