RAG (Retrieval-Augmented Generation) systems often "hallucinate" not due to the model, but by being fed incorrect context. This article highlights that prompt engineering alone is insufficient, advocating for "context engineering" across four critical "bricks": document parsing, question parsing, retrieval, and generation. A naive RAG pipeline fails at each of these stages when tested on real-world documents from NIST and the World Bank.

Document parsing fails when tables are flattened into noise, losing the crucial alignment between labels and data. Question parsing falters when documents use synonyms not present in the query, preventing relevant sections from being found. Retrieval struggles with structure, often missing defining information that falls below a frequency-based cutoff. Finally, generation can confidently provide incorrect answers when the source document lacks the requested information, as seen with a missing 2026 forecast.

The proposed solution involves upgrading each brick with tighter contracts. Relational parsing preserves document structure, question parsing expands queries with domain synonyms, retrieval routes based on document structure (like tables of contents), and generation is improved through better context. These improvements, demonstrated with a runnable notebook, address the root causes of RAG inaccuracies, moving beyond simple prompt tweaks to robust context management.