A developer building a CSV Q&A system discovered that larger context windows don't improve RAG accuracy for data aggregation—they make errors harder to spot. When testing internal demos, the author trusted a response showing grocery spend of $1,140,033.24, but manual verification revealed the RAG-generated breakdown was less than half correct.

The fundamental problem is that RAG pipelines flatten structured data into plain text and retrieve only partial slices. For queries requiring SUM, GROUP BY, or COUNT operations across entire datasets, the LLM performs pattern-matching on incomplete data rather than actual computation. This creates dangerously misleading outputs that appear authoritative but omit 92% of the underlying information.

To quantify this issue, the author benchmarked retrieval-based pipelines against a deterministic full-scan engine across 100,000 rows and seven query types. Results showed that as context increased from 5 rows to 8,000 rows, error detectability collapsed—from obviously wrong to nearly impossible to catch, despite the answer remaining 50% incorrect.

The solution routes computation queries away from RAG entirely, using semantic engines that execute exact aggregations. Code is available on GitHub. This work exposes a critical flaw in treating retrieval systems as calculation engines for structured data analysis.