Over the last year, AI retrieval architectures have become a hot topic in analyst reports and technical blogs. I built four different approaches on a laptop to understand whether adding a knowledge graph materially improves answer quality. The dataset was deliberately small, consisting of two recently published Anthropic articles on AI Security and a set of evaluation questions. My goal wasn't to identify a universally superior architecture, but to understand how different retrieval approaches behave with the same information.

To keep the comparison fair, I used Microsoft's Phi-4 language model (14 billion parameters) for answer generation, running it locally on a Thinkpad through Ollama. The all-Mini LM-L6-v2 embedding model powered semantic search. For knowledge graph storage and queries, I used Neo4j to manage entities and relationships extracted from source material. A frontier model approach using Claude Haiku 4.5 provided additional comparison.

The outcome was clear: the frontier model produced the strongest answers overall. However, Graph RAG consistently outperformed traditional RAG on certain question types. The results highlighted that the value of a graph depends heavily on the specific problem being solved. This experiment provides practical insights into when the additional complexity of Graph RAG is justified and when traditional approaches may suffice.