Governance & testing
What is an ontology, and why version it?
Your first vocabulary will be wrong, because nobody knows their own taxonomy before extracting it. The question is whether changing it is cheap.
10 min read
An ontology, in a retrieval system, is the list of entity types and relationship predicates the extractor is allowed to use. It is a small object with outsized consequences, and the single most important fact about it is that your first version will be wrong.
Why the first version is always wrong
You have to give the extractor a vocabulary before it reads your corpus. That means you are guessing at your own taxonomy from memory, and nobody knows their own taxonomy from memory. The words your content actually uses, the distinctions it actually draws, and the relationships that actually recur are all discoveries rather than recollections.
This is not a criticism of anyone's planning. It is a structural property of the problem, which means the useful question is not "how do I get the vocabulary right first time" but "what does it cost me to change it".
What it costs in most systems
Suppose you decide that "topic" and "concept" were always the same thing and you want one term instead of two. In a typical implementation your options are:
| Option | Cost |
|---|---|
| Delete entities with the old type | Destroys extraction work, and the facts attached to them |
| Re-run extraction with the new vocabulary | Model spend over the whole corpus, and hours to days of processing |
| Retype the rows in place | Requires knowing every affected row; irreversible if wrong |
| Leave it and accept both terms | The graph now has two names for one thing, permanently |
All four are bad enough that the real-world outcome is predictable: nobody changes the vocabulary. It freezes at the first guess. The corpus grows past it, extraction quality quietly declines, and there is no metric anywhere that says "our taxonomy stopped matching our content".
What makes change cheap
The fix is a vocabulary change that does not touch what was already extracted. Whatever the new vocabulary does not admit drops out of view — it is not deleted and it is not retyped. Three properties are the test of any design that claims this, and they are the whole argument:
- Reverting is possible. Applying the previous version brings everything back whole, because nothing ever left.
- The extractor never re-runs. A vocabulary change costs no model calls over your corpus — it is not a re-processing job at all.
- You can preview the impact exactly. Before applying, count how many entities and assertions each change would drop from view. That turns "should we rename this term" from a judgement call into an arithmetic one.
Any design that fails one of those three is a migration with extra steps, and teams treat migrations the way they treat every other irreversible operation: they avoid them.
Versioning: live and curated
Once change is cheap, a useful pattern becomes available: keep two kinds of vocabulary at once.
One is live and always learning — every new type the extractor discovers is registered into it, forever. It is not your curated taxonomy; it is the honest record of what your content contains. It should be read-only to humans, because its value is that nobody has tidied it.
The other is a curated version, forked from live at a point in time, edited deliberately, and frozen once applied. This is the vocabulary your team stands behind.
The interesting thing is the gap between them, which grows on its own.
Drift: the reason any of this works
Drift is what the live vocabulary has discovered since you forked your curated one. It is a list, and it is the answer to the question nobody can otherwise answer: has our taxonomy gone stale?
Without it, curating a vocabulary is an act of faith. You freeze something reasonable and find out eighteen months later that half your recent content is being typed into terms you deprecated. With it, staleness is a number you can look at, and folding the new discoveries in is a new draft rather than an archaeology project.
This is also the argument for keeping the live version messy. A vocabulary that has been tidied cannot tell you what it discovered, because someone already decided what was worth keeping.
The operations you actually need
Five, and they map to the five things people actually want to do:
| Operation | What it does | What it preserves |
|---|---|---|
| Add | Offer a new term to the extractor | Everything — it is forward-looking only |
| Rename | Change a term's name | The old name as an alias, so what was already extracted still resolves |
| Merge | Fold two terms into one | Every entity — nothing extracted is rewritten, so it can be undone |
| Deprecate | Stop offering a term to the extractor | Everything already extracted under it |
| Remove | Hide what a term typed from the active view | Everything, so it is reversible |
The pattern across all five: none of them rewrites an entity. Rename keeps an alias, merge can be undone, remove only hides. That is what makes the whole set safe to use.
What to take away
- Your first vocabulary will be wrong. Design for changing it, not for getting it right.
- Make a vocabulary change reversible. Hide, never delete or retype.
- Keep a live auto-learning version and a curated frozen one.
- Measure drift between them. It is the only signal that your taxonomy is going stale.
- Preview impact per term before applying. Arithmetic beats judgement here.