What is an ontology, and why version it?

An ontology, in a retrieval system, is the list of entity types and relationship predicates the extractor is allowed to use. It is a small object with outsized consequences, and the single most important fact about it is that your first version will be wrong.

Why the first version is always wrong

You have to give the extractor a vocabulary before it reads your corpus. That means you are guessing at your own taxonomy from memory, and nobody knows their own taxonomy from memory. The words your content actually uses, the distinctions it actually draws, and the relationships that actually recur are all discoveries rather than recollections.

This is not a criticism of anyone's planning. It is a structural property of the problem, which means the useful question is not "how do I get the vocabulary right first time" but "what does it cost me to change it".

What it costs in most systems

Suppose you decide that "topic" and "concept" were always the same thing and you want one term instead of two. In a typical implementation your options are:

All four are bad enough that the real-world outcome is predictable: nobody changes the vocabulary. It freezes at the first guess. The corpus grows past it, extraction quality quietly declines, and there is no metric anywhere that says "our taxonomy stopped matching our content".

The design that makes change cheap

The fix is to make schema conformance a computed property rather than a destructive operation. Concretely: every entity and every assertion carries a flag for whether the currently active vocabulary admits it, and applying a new vocabulary recomputes that flag rather than modifying the rows.

A row the new schema does not admit is filtered out of the active view. It is not deleted and it is not retyped. Three consequences follow, and they are the whole argument:

• Reverting is possible. Applying the previous version recomputes the flags and the rows come back, because they never left.
• The extractor never re-runs. A vocabulary change costs a background sweep over existing rows, not model calls over your corpus. Time, not money.
• You can preview the impact exactly. Before applying, count how many entities and assertions each change would drop from the active view. That turns "should we rename this term" from a judgement call into an arithmetic one.

entities
  id, normalized_name, type, in_schema  ← recomputed on apply

assertions
  id, subject, predicate, object, in_schema  ← same

-- applying a version is a sweep, not a migration
UPDATE entities
   SET in_schema = (type = ANY($1))   -- $1 = admitted types
 WHERE workspace_id = $2;

-- so reverting is the same sweep with the old type list

Versioning: live and curated

Once change is cheap, a useful pattern becomes available: keep two kinds of vocabulary at once.

One is live and always learning — every new type the extractor discovers is registered into it, forever. It is not your curated taxonomy; it is the honest record of what your content contains. It should be read-only to humans, because its value is that nobody has tidied it.

The other is a curated version, forked from live at a point in time, edited deliberately, and frozen once applied. This is the vocabulary your team stands behind.

The interesting thing is the gap between them, which grows on its own.

Drift: the reason any of this works

Drift is what the live vocabulary has discovered since you forked your curated one. It is a list, and it is the answer to the question nobody can otherwise answer: has our taxonomy gone stale?

Without it, curating a vocabulary is an act of faith. You freeze something reasonable and find out eighteen months later that half your recent content is being typed into terms you deprecated. With it, staleness is a number you can look at, and folding the new discoveries in is a new draft rather than an archaeology project.

This is also the argument for keeping the live version messy. A vocabulary that has been tidied cannot tell you what it discovered, because someone already decided what was worth keeping.

The operations you actually need

Five, and they map to the five things people actually want to do:

The pattern across all five: none of them rewrites an entity. Rename keeps an alias, merge records a fold, remove sets a flag. That is what makes the whole set safe to use.

One practical warning: normalisation

Conformance is an exact string match, so the stored form of a term is what matters, and it is not what someone types. "Record Label" becomes record_label. Predicates are usually lemmatised too, so "Was A Member Of" becomes member_of.

Show the stored form in whatever editor people use. Discovering the transformation after building a taxonomy around the wrong assumption is a genuinely annoying afternoon.

What to take away

1. Your first vocabulary will be wrong. Design for changing it, not for getting it right.
2. Make conformance a computed flag. Filter, never delete or retype.
3. Keep a live auto-learning version and a curated frozen one.
4. Measure drift between them. It is the only signal that your taxonomy is going stale.
5. Preview impact per term before applying. Arithmetic beats judgement here.
6. Surface the normalised form to whoever edits it.