Strictly speaking, the statement “LLMs are next-token predictors” isn’t wrong, but it’s incomplete. It’s a fine zeroth-order approximation, and it is grounded in something real: transformer-based language models emit tokens autoregressively. During pre-training, the model repeatedly takes some prior tokens, looks at the token that actually followed them, and makes that token more likely to be sampled next.

Crucially, every actual_next_token comes from an existing sequence in training_data. It’s probably fair to say that the base model also behaves as a next-token predictor. But the LLMs we use are not just base models. They are post-trained, and a key part of modern post-training is reinforcement learning with verifiable rewards (RLVR). During pre-training, the model learns only from sequences that already exist in the training data. During RLVR, the model explores by generating new sequences and learning from their outcomes.

A chess analogy makes this distinction easier to see. The first system is trained on a large database of grandmaster games and predicts the move a grandmaster would most likely play next. The second is an idealized chess engine that has explored every possible game and chooses the move that leads to the highest probability of winning. Calling the second system a “next-move predictor” would be strange.

Other post-training techniques matter too. Reinforcement learning from human feedback (RLHF) shifts the model toward simulating a helpful assistant. RLVR goes further still: it allows an LLM to explore and learn from its own generated sequences.