LLM systems face high costs from network latency, token charges, and compute overhead. Caching offers a direct solution to these expenses. The two primary methods are exact-match caching, which stores responses for identical prompts, and semantic caching, which matches semantically similar queries using embeddings. Both aim to bypass costly model re-runs.

Exact-match caching uses a deterministic key from the prompt and metadata to retrieve stored responses. It's simple and safe but brittle for natural language. Semantic caching requires embedding models and vector stores to find similar prompts, enabling higher hit rates for conversational queries. Each method has distinct trade-offs in implementation complexity and accuracy.

Choosing the right approach depends on the use case. Deterministic, templated systems favor exact-match caching. For open-ended queries, semantic caching provides better reuse. Many production systems combine both, trying exact-match first and falling back to semantic search. Monitoring cache hit rates and false positives is critical for maintaining performance and cost savings.