Retrieval-Augmented Generation (RAG) often fails when conversation history accumulates, not due to poor retrieval, but due to uncontrolled context stuffing. A developer built a pure Python context engineering system to explicitly manage memory, compression, and token budgets, addressing the gap most tutorials ignore.

When context grows, naive RAG systems drop relevant documents or overflow the prompt, causing models to forget recent turns. This custom layer sits between retrieval and prompt construction, making architectural decisions about what information actually makes it into the LLM’s working memory, a concept Andrej Karpathy recently termed context engineering.

The new pipeline integrates hybrid retrieval (blending keyword and TF-IDF), a heuristic re-ranker that boosts documents based on internal tags, and memory management using exponential decay. This decay method allows older conversational turns to fade gracefully rather than abruptly dropping out, preventing noise accumulation.

Benchmarked on CPU-only hardware, this system demonstrates measurable improvements over basic RAG setups for multi-turn chatbots. The implementation provides tangible control over the context window, a necessity when production constraints negate the idea of unlimited token space, proving context control is essential for production AI agents.