This article explores a "LLM cascade" strategy for Retrieval-Augmented Generation (RAG), optimizing for both cost and accuracy when extracting typed fields from documents. Instead of sending every field to an expensive flagship model like GPT-4, the approach starts with a cheaper, local model. If this initial model fails to provide accurate output, the task is escalated to a more powerful model.

This "loop engineering" approach is supported by a benchmark involving twenty local models against a hosted flagship. Key findings indicate that model size is a poor predictor of performance; smaller models like Qwen3:4b and Mistral:7b outperformed larger ones in some cases. Furthermore, the prompt content, specifically including business vocabulary and definitions, proved to be a more significant factor in improving accuracy than simply increasing model size.

The cascade prioritizes cost savings by defaulting to cheaper models but ensures correctness through a validation loop. If the initial, low-cost model's output is not validated, it escalates to a stronger model, preventing errors from shipping. This method is integrated into the document intelligence pipeline, leveraging similar "start cheap, escalate on demand" principles for parsing and generation.