This article introduces "Loop Engineering for RAG Generation," a method to optimize how retrieved documents are fed to a generation model. Traditional Retrieval-Augmented Generation (RAG) pipelines often send all top-K retrieved candidates to the Large Language Model (LLM) simultaneously, which can be inefficient for simple questions.

Two regimes are proposed: batch processing, where all K candidates are sent at once (ideal for listing or comparison questions), and sequential processing, where candidates are sent one by one, with the LLM assessing sufficiency after each. The sequential approach can cut token costs by up to 80% for factual questions where the top-1 candidate suffices. This method leverages a "sufficiency signal" from the LLM, indicated by `answer_found` and `complete_answer_found` booleans, to determine when to stop the loop.

The choice between batch and sequential processing is made per question, guided by a question parser that analyzes question shape and intent. This dispatching system ensures efficiency across various question types and domains, such as insurance, legal, and medical documents. The article provides runnable code and details the `Answer With Evidence` schema that facilitates this process.