What is graph-enhanced retrieval?

Graph-enhanced retrieval means running a knowledge graph alongside vector search and using it to reach documents the query's own wording would never have found. It is a specific technique with a specific cost, and it is worth understanding both before adopting it — including the cases where it buys you nothing.

The problem vector search cannot solve

Vector search retrieves passages that resemble the question. That is a good match for most questions, and it fails in one specific way: when the answer requires combining two passages that do not resemble each other.

Consider a question like "does finishing onboarding early reduce churn?" One document reports retention rates by onboarding speed. Another explains that accounts which reach value quickly stay. Neither document mentions the other. A vector search over the question will probably find both, because both are topically close — but the connection between them is not written down in either one, so the answer either restates both passages and leaves the reader to join them, or it asserts a connection the model inferred and cannot cite.

That second outcome is the one to worry about. It looks like a good answer. It is a hallucination with two real citations attached.

What the graph adds

A knowledge graph built from your corpus stores what documents assert as edges between entities. When two documents each assert something about the same entity, that entity becomes a join point, and the relationship between the two documents becomes a thing you can query rather than a thing a model has to guess.

In practice this shows up as two additional retrieval passes:

• Expansion. Extract entities from the question, resolve them to graph nodes, traverse a couple of hops, then search again using the query plus the entity names you reached. This finds documents whose vocabulary does not overlap with the question at all — ask about React and you reach material that only ever says JSX.
• Evidence. Take the facts the traversal crossed, find every document that provides provenance for them, and search again restricted to that set. This is the pass that finds a passage supporting a claim rather than a passage resembling a phrase.

Both passes are additive. The direct vector search still runs, at full weight, and if the graph contributes nothing the result is a normal vector search rather than an error. That property is worth insisting on when you build or buy: a graph that can degrade your baseline is worse than no graph.

How the results get combined

Three retrieval passes produce three result lists with three incompatible score scales. A cosine similarity of 0.7 and a keyword relevance of 12.4 cannot be meaningfully added, and normalising them requires inventing a conversion factor that will be wrong for some corpus.

The standard answer is reciprocal rank fusion, which throws the scores away and uses only the orderings. Each document scores the sum over lanes of 1/(k + rank), for some small constant k. A document ranked first in two lanes beats a document ranked first in one, and no lane can dominate because its numbers happen to be larger.

What it costs

Three costs, all real:

The ingest cost is trivial in money and significant in operational surface: extraction is the step that fails, retries, and needs concurrency limits. The query-time cost is latency you are adding to every search, which matters a great deal for a typeahead and not at all for an analytical question.

When it adds nothing

Be honest with yourself about these, because the technique is fashionable and the failure is quiet.

• Your questions are answered by single passages. "What is our refund window" is a lookup. A graph adds latency and cost to a problem that was never about connecting documents.
• You have one content source with one vocabulary. The graph earns its keep by bridging entities across sources. With one source there is nothing to bridge, and the metric that would tell you it is working — entities appearing in two or more systems — reads zero.
• Your corpus is small. Under a few thousand documents, a vector search over everything is close to exhaustive already. The graph is solving a recall problem you do not have.
• Latency is your binding constraint. If the search box has to feel instant, the extra passes are not available to you regardless of what they would add.

How to tell whether it is working

Two metrics, neither of which is "does it feel better".

The first is how often graph expansion contributes anything to the fused result at all. If the expanded and evidence lanes rarely surface a document the direct lane missed, the graph is decoration. On our own evaluation corpus, fusion involves graph-derived documents on 201 of 231 question runs — so the lanes are doing work.

The second is harder and more important: does it answer questions the baseline could not? This requires a question set with known answers and a run of each system over it, counting questions answered by one and refused by the other in both directions. It is more work than it sounds, and it is the only measurement that distinguishes a real improvement from a system that has simply become more willing to talk.

What to take away

Graph-enhanced retrieval is the right technique when your content spans multiple systems, your questions span multiple documents, and you need to be able to point at the evidence behind an answer. It is the wrong technique when you have a lookup problem, one source, or a latency budget measured in tens of milliseconds.

And if you adopt it, insist on two properties: that the vector baseline always runs, and that every fact the graph holds can name the documents that assert it. Without the first you have added a way to get worse. Without the second you have added a way to be confidently wrong.