Building a knowledge graph for retrieval

A knowledge graph built for retrieval is a different thing from a knowledge graph built for reasoning. You are not trying to model the world correctly; you are trying to store what your documents assert, with enough bookkeeping that you can find your way back to them. That difference simplifies some decisions and makes others harder.

Three primitives

You need exactly three things. Systems that need more are usually solving a modelling problem rather than a retrieval one.

• Entity — a thing mentioned in your content. Identity matters enormously and is the subject of most of this guide.
• Assertion — one edge per fact: a subject, a predicate, an object. One edge, not one per mention.
• Provenance — one row per (assertion, document) pair, recording that this document asserts this fact and how confident the extraction was.

The temptation is to collapse the third into the second by putting a document identifier on the edge. Resist it; that design breaks the second time two documents agree, and the breakage is subtle enough to survive review.

Entity identity is the whole problem

What makes two mentions the same entity? The workable answer for retrieval is a triple: the tenant, a normalised form of the name, and the type. Same tenant, same normalised name, same type means same node.

Normalisation is where the judgement lives. Lowercasing and collapsing whitespace is uncontroversial. Replacing spaces with underscores makes the stored form stable. For predicates it is usually worth lemmatising as well, so "reduces", "reduced" and "reducing" do not become three relationships.

"Record Label"        → record_label
"record  label"       → record_label
"Learning Objective"  → learning_objective

predicates, additionally lemmatised
"Was A Member Of"     → member_of
"reduces"             → reduce

Whatever you choose, surface the stored form to whoever is editing the vocabulary. Conformance is an exact string match, and someone typing "Record Label" needs to know it becomes record_label before they build a taxonomy around the wrong assumption.

Resolution: three stages

Normalised exact match catches most mentions. It will not catch typos, near-synonyms, or two teams that use completely different words for the same concept. So resolution escalates:

1. Exact match on the normalised name and type. Cheap, and handles the large majority.
2. Trigram similarity. Catches typos, plurals, stray punctuation — cases where two strings clearly mean one thing. Postgres has this built in; most stores have an equivalent.
3. Embedding similarity. The last resort, for mentions that mean the same thing and share no characters. Expensive, and the one most likely to produce a false merge, so it wants a conservative threshold and a review queue for the near-misses.

The review queue matters more than it sounds. Automatic merging at a low threshold quietly folds distinct entities together, and the symptom — answers that conflate two things — is very hard to trace back to a resolution decision made three months ago. Keep the borderline cases and let a human look.

The concurrency bug

This is the failure that everyone building this hits, and it does not announce itself.

Sequential extraction over a real corpus takes far too long, so you make it concurrent. Two workers now process two documents that both mention single sign-on. Both look for an existing entity. Both find none, because neither has committed yet. Both create one.

The fix is that the lookup and the insert must not be separable — resolution runs inside a transaction, with a uniqueness constraint on the identity triple as the backstop. When two workers race, one wins and the other reads the winner's row. It is a small amount of code and it is not optional.

Assertions: one edge per fact

Every mention of a fact should strengthen one edge rather than create another. If three documents say onboarding reduces churn, you want one edge with three provenance rows, not three edges.

The reason is that traversal weights and corroboration counts both become meaningless otherwise. Three edges between the same pair of nodes makes that relationship look three times as connected as it is, which distorts any path scoring you do. And "how many documents support this" — the single most useful number a retrieval graph can give you — is only answerable if the edge is unique.

One caveat worth knowing before you trust your own counts: predicate variants defeat this. If one document says "reduces" and another says "lowers", you get two edges regardless, because the predicate is part of the identity. Lemmatising helps; it does not solve it. On our own corpus about eighteen percent of edges are predicate variants of another edge, which means corroboration counts should be read as a floor rather than a measurement.

What NOT to store

Three things it is tempting to add and better to leave out.

• Inferred facts, mixed in with asserted ones. If you derive an edge from other edges, mark it as derived. Otherwise you eventually cite an inference to a document that never said it, which is the exact failure the provenance table exists to prevent.
• Hand-authored entities. An entity a person typed in has no provenance, and a fact with no document behind it cannot be cited. If you need editorial facts, keep them somewhere else and label them.
• Rich attribute payloads on the node. It is very tempting to hang metadata off entities. Retrieval does not read it, it drifts out of sync with the documents, and it turns your graph into a database with a maintenance problem.

The graph should be disposable

The most useful architectural decision available here: make the relational store the source of truth and treat the graph as a projection that can be rebuilt from it.

This buys three things. A graph store failure is a rebuild rather than a data-loss event. Changing your vocabulary is a recomputation rather than a re-extraction, so it costs time instead of model spend. And you can swap the graph backend, which in practice means small deployments can traverse with recursive SQL over the same database and never operate a second datastore at all.

relational store           ← source of truth
  documents, chunks
  entities, assertions
  assertion_sources        ← provenance
  vocabulary versions

vector index               ← projection
graph store                ← projection
                             (or recursive CTEs
                              over the same DB)

The cost is that you write every extraction result twice, once to the relational tables and once to the projections. That is a small amount of write amplification in exchange for never having to re-run a model call because a store lost a collection.

A short checklist

1. Identity is a triple: tenant, normalised name, type. Enforce it with a constraint.
2. Resolution escalates exact → trigram → embedding, transactionally, with a review queue for borderline merges.
3. One edge per fact. One provenance row per (edge, document), each with its own confidence.
4. Mark derived edges as derived. Never store an entity a person typed.
5. Relational store is truth; the graph is rebuildable.
6. Surface the normalised form to whoever edits the vocabulary.

None of this is exotic. All of it is the difference between a graph that answers questions after a year and one that has quietly filled with duplicates nobody can account for.