Every agent already runs a loop. Loop engineering adds a loop around the agent itself, enabling it to evaluate its output, try again when the work falls short, and refine its instructions when the same mistakes recur. Today, you perform that role: reviewing the work, diagnosing what went wrong, and prompting the agent again. This article shows how to automate that process with a working example, while exploring where human judgment still belongs.

A 70-billion-parameter model requires reading roughly 140 GBs of weights out of the GPU memory. On a modern data center GPU, this transfer can take tens of milliseconds. The actual calculation applied to these weights takes a fraction of that time. This means that the processor’s math units are unused for most of the time taken by the token generation step. Speculative decoding is a technique that converts this unused capacity into output.

A second, much smaller model produces several candidate tokens in advance. The large model evaluates all of them in a single forward pass instead of one pass per token, resulting in 2-3 times faster generation. To make things better, the text produced remains statistically identical to the output of the large model running alone.

Text generation works one token at a time. The model reads everything produced so far, computes a probability distribution over its vocabulary, selects the next token, appends that token to the input, and repeats the cycle. Each cycle is called a forward pass, and every forward pass runs the input through all layers of the model. Modern inference systems use a KV cache, which stores the attention state for tokens already processed, cutting the work done inside each pass, though the requirement for one pass per token still remains.