Core concepts
Graph RAG vs vector RAG
A straight comparison: what each one retrieves, what each one can state, where the latency goes, and how to decide without running a bake-off.
10 min read
Vector RAG retrieves passages that resemble the question. Graph RAG does that too, and additionally retrieves passages connected to the question through facts extracted from your corpus. The second is a superset of the first, which makes the comparison less about quality and more about whether the extra work pays for itself on your content.
What each one actually retrieves
| Vector RAG | Graph RAG | |
|---|---|---|
| Matches on | Semantic similarity to the question | Similarity, plus what the content is connected to |
| Finds documents sharing no vocabulary with the query | Rarely | Routinely |
| Can enumerate the documents behind a claim | No | Yes |
| Needs a model call at ingest | No — embedding only | Yes — extraction per document |
| Added query latency | None | Some — corpus dependent |
The one question type that separates them
Almost every question either system handles equally. The separation happens on questions whose answer requires a relationship that no single passage states.
Take "do I need the security module before the deployment lab?" The lab document lists a prerequisite: familiarity with role-based access control. The security course teaches role-based access control. Neither document mentions the other course by name.
Vector RAG will likely retrieve both, because both are topically adjacent to the question. What it cannot do is state that one is a prerequisite for the other, because that sentence does not exist in the corpus. So the model either hedges, or it asserts the connection and cites two documents that do not actually say it.
Graph RAG has an edge — deployment lab, prerequisite, role-based access control — that a document did assert. The connection is retrieved rather than inferred, and the answer can state it while citing the documents that each hold half.
Where the latency goes
Worth being concrete, because "graph RAG is slower" is not actionable. A search that writes an answer spends its time in three places: the ordinary search underneath, which is identical to vector RAG and runs regardless; whatever the graph adds; and writing the answer, which is the same in both systems.
The useful observation is that writing the answer usually dominates. If you are generating prose at all, the graph overhead lands on a number that was already large. If you are only returning passages, the overhead is proportionally much larger and the calculus changes — so the surface matters more than the corpus does. Measure all three on your own content rather than taking anyone's figures, including ours.
What graph RAG does not fix
Three things people expect it to solve and it does not.
- Content that does not contain the answer. If nobody wrote it down, no retrieval architecture will find it. This is by far the most common cause of bad answers, and it is a content problem.
- Bad chunking. If your chunk boundaries cut claims in half, both systems retrieve half-claims. The graph will faithfully extract facts from mangled text.
- Ranking preferences. If your users want the most recent document and your retrieval prefers the most similar one, that is a ranking policy question. A graph does not know what you consider authoritative.
How to decide without running a bake-off
A proper evaluation is expensive — you need a question set with known answers and a scoring rubric. Before committing to that, three cheap signals get you most of the way.
- Count your content sources. One source with one vocabulary means the graph has nothing to bridge. Two or more, using different words for the same concepts, is where it earns its keep.
- Sample twenty real user questions and mark each one: answered by a single passage, or requiring two. If almost all are single-passage, you have a lookup problem and vector RAG is sufficient.
- Ask whether anyone will need to justify an answer. If a wrong answer is embarrassing rather than expensive, the evidence trail is a nice-to-have. If someone will be asked "where did this come from", provenance is the feature you are actually buying.
An honest note on the accuracy claims
Graph RAG is currently fashionable, and a lot of published comparisons are not measuring what they claim. The specific trap is counting answer rate: a system that answers more questions looks better on a naive metric, even if some of the extra answers are wrong.
The correction is to include questions your corpus genuinely cannot answer, and score refusal as correct on those. Without that control, "answers more" is indistinguishable from "grounds better", and the more talkative system wins regardless of whether it is right.
The short version
- Choose vector RAG for lookup questions, a single source, tight latency budgets, and small corpora.
- Choose graph RAG for questions that span documents, content spread across several systems, and any situation where an answer has to be defensible.
- Whichever you choose, insist that the vector baseline always runs. A graph that can degrade your floor is a liability.
- And measure with refusal controls, or you will measure the wrong thing.