Buy or build: grounded answers over your own content

Most teams that decide to build this are not wrong to think they can. The first version genuinely does take a week. What follows is an account of what arrives after that week — written by people who built it, and who would rather you made this decision with the real inventory in front of you than discover it in month four.

The week-one version

Chunk your documents, embed the chunks, put the vectors in a store, embed the query, retrieve the nearest neighbours, stuff them into a prompt. This works. It demos well. On a few thousand documents with one person using it, it is genuinely hard to distinguish from a mature system.

That is the trap, and it is not a trap about quality. It is a trap about which problems have shown up yet. Everything below is a problem that only appears with concurrency, scale, a second content source, or time.

What arrives with the second source system

The single-source version of this problem is a search box. The multi-source version is a different product, and the difference is entity resolution.

Your help centre calls it "SSO". Your tickets call it "single sign on", "single-sign-on", and occasionally "SAML". Your onboarding course calls it "identity federation". A vector search will partly paper over this, because embeddings are tolerant of surface variation. A graph will not: you get four nodes, none of which knows about the others, and the traversal that was supposed to connect your content instead fragments it.

So you build resolution. Exact match on a normalised form catches most of it. Trigram similarity catches typos and plurals. Embedding similarity catches the cases that share no characters at all. Three stages, escalating in cost, and you will get there in an afternoon.

Then you make extraction concurrent, because sequential extraction over a real corpus takes days, and the whole thing breaks in a way that is genuinely hard to see.

The fix is that resolution has to be transactional — the check and the insert cannot be separated by a window another worker can drive through. That is a straightforward thing to say and a fiddly thing to get right, and it is the first place where the week-one version stops being a week.

What arrives with provenance

The naive graph stores facts: subject, predicate, object. It works until someone asks the obvious question — which document said that? — and you discover you did not keep it.

So you add a document reference to the edge. Then two documents assert the same fact, and you have to decide: two edges, or one edge with two references? Two edges means your traversals double-count and your corroboration numbers are meaningless. One edge with a set of references means you need a join table, and now you have to decide what happens when one of those documents is deleted.

The correct answer, and it takes a while to arrive at, is one row per (edge, document) pair, each carrying its own confidence, and a rule that a fact is retracted only when its last supporting document stops asserting it. Every simpler design has a case where deleting one document out of four either destroys a fact that three documents still support, or leaves a fact standing that nothing supports any more.

What arrives with the ontology

You have to tell the extractor what to look for. So you write a list of entity types, and because you write it before you have seen what your corpus contains, it is wrong. That is not a criticism — it is unavoidable. Nobody knows their own vocabulary until they have extracted it.

The question is what happens when you want to change it. In most implementations the answer is: nothing good. Changing an entity type means the entities already typed with the old one are wrong, and your options are to delete them, to retype them with another model pass, or to leave them and accept an inconsistent graph.

All three are bad enough that the actual outcome is predictable: nobody changes the vocabulary. It freezes at whatever the first guess was, the corpus grows past it, answers get quietly worse, and the reason is invisible because there is no metric for "our taxonomy stopped matching our content".

The design that avoids this is to make conformance a computed property rather than a destructive one. Applying a new vocabulary marks rows as in or out of schema and filters the view; it does not delete or retype anything. Reverting is then applying the previous version and letting the sweep run again, and the model never re-runs. It costs a boolean column and a background job, and it is the difference between a vocabulary your team maintains and one they are afraid of.

What arrives with orchestration

This is the part that took us longest and the part that is easiest to underestimate, so it is worth being specific with numbers rather than adjectives.

Our first production pipeline lost nine documents out of two hundred and five. Not failed loudly — lost. They were accepted, they went into a job, the job died somewhere in the middle, and nothing retried. The user saw a successful POST and then no document.

Its worst single step took three hundred and forty-eight percent of the load balancer's timeout budget, which is a fancy way of saying the request was killed before the work finished, every time, and the work was then abandoned. Extraction at the ninetieth percentile took about a hundred and three seconds per document. Throughput was three documents a minute.

What fixed it was not faster code. It was retries at the step rather than the job, so a single failed shard does not take a document with it; durable waits, so a run can pause for a rate limit instead of dying; concurrency ceilings per tenant, so one backfill cannot starve everyone else; and replayable runs, so a bad deploy is recoverable rather than a data-loss event.

None of that is novel. All of it is work, and all of it is work you would be doing rather than working on your product.

What it actually costs

The model spend is the part everyone worries about and it is the cheap part. Extraction runs about two thousand tokens per document, which lands between $0.00019 and $0.00038 depending on the model. A ten thousand document corpus is about two dollars. A million documents is about a hundred and ninety.

• Model spend: negligible until you are in the millions, and even then it is a line item rather than a decision.
• Datastores: a vector index, and usually a graph database. Both need sizing, backups, monitoring and a rebuild path. This is real money and real operational surface.
• The engineer: one person who understands the whole pipeline and is on the hook when a document does not show up. This is the actual cost, and it does not end.

For reference, our own infrastructure runs a few thousand dollars a month at a modest production shape. That is a reasonable floor for a comparable build, before anyone's salary.

The four questions that decide it

Everything above is context. These are the questions that actually determine the answer, and two of them point at building.

1. Is retrieval quality what your customers buy from you? If your product IS search, or if a percentage point of relevance is your differentiator, build it. You will out-tune any general-purpose service on your specific corpus, because you can afford to care about it more than we can.
2. Do you have a hard deployment or model constraint? Your own cloud, a specific region, a specific model, customer-managed keys. If any of these is non-negotiable, build — we offer one managed deployment in one region on models we choose, and no amount of goodwill changes that this quarter.
3. Can you staff the pipeline indefinitely? Not build it — own it. Through the person who wrote it leaving, through the model deprecation, through the day a store loses a collection. If the honest answer is that it becomes nobody's job after launch, buying is the lower-risk path.
4. Does your content span more than one system? If it does, the expensive parts above are all in your path: resolution across vocabularies, provenance across sources, a vocabulary that needs governing. If it does not — one source, one taxonomy — the week-one version may genuinely be enough, and you should not buy a graph to solve a problem you do not have.

What we would tell you on a call

That the free tier exists precisely so this decision does not need a meeting. Ten thousand documents, cited answers, no card. Point it at a real slice of your content and ask it real questions. An afternoon of that is worth more than any comparison document, including this one.

And that if the answers are not useful, the graph will not rescue them. When retrieval fails on a corpus it is usually because the content does not contain the answer, and no amount of infrastructure fixes that. Finding that out in an afternoon on a free tier is a good outcome — it is a content problem, and now you know.