An interactive guide that walks users through the mechanics of large language models has landed on Hacker News. Built from the transcript of Andrej Karpathy’s “Intro to Large Language Models” lecture, the site was generated with Claude Code and packs a complete walkthrough—from raw web crawl to chat‑ready assistant—into one HTML file. It also offers live tokenizers for GPT‑4, Claude and Llama.

Data collection starts with Common Crawl, which has indexed about 2.7 billion pages since 2007. After aggressive filtering—removing malware, low‑quality sites, non‑English pages and duplicates—the pipeline yields roughly 44 TB of high‑quality text, equivalent to 15 trillion tokens, supporting today’s AI research. This curated corpus, dubbed FineWeb, feeds the tokenization and training stages that define modern LLM performance.

Training employs a Transformer initialized with billions of parameters; contemporary models like Llama 3 sport 405 B weights and ingest the 15 trillion‑token dataset. Each step predicts the next token, nudging parameters via gradient descent, while temperature controls sampling randomness during inference. By exposing the full pipeline in an interactive format, the guide demystifies how raw text becomes the conversational AI behind services such as ChatGPT.