How to add grounded search to your product

The hard parts of adding grounded search to an existing product are not the retrieval. They are five integration decisions that are cheap to make correctly at the start and expensive to change later. This guide is those five, in the order you will hit them.

1. How you key documents

Whatever retrieval system you use, it will let you supply your own identifier for each document. Treat that identifier as the most important integration decision you make.

It should be stable, unique within its source, and derivable from your own data without a lookup. A primary key works. A slug works if slugs never change. A hash of the content does not work, because the whole point is that the identifier survives the content changing.

Get this right and your ingest path becomes idempotent for free: sending a document again replaces the previous version rather than adding a second copy, so retries are safe, webhooks can fire twice, and a full backfill is a loop rather than a migration with a cleanup phase.

// good — stable, unique, derivable
externalId: `article-${article.id}`
externalId: `ticket-${ticket.number}`
externalId: `course-${courseId}-lesson-${lessonId}`

// bad
externalId: hash(article.body)   // changes when content does
externalId: article.title        // not unique, and it changes
externalId: crypto.randomUUID()  // a new document every sync

2. Where the tenancy boundary goes

If your product is multi-tenant, decide early whether one of your customers maps to one workspace (or index, or namespace — whatever the isolation primitive is called) or whether you are going to filter by a tenant field on every query.

Pick the isolation primitive. Filtering works until the day someone writes a query path that forgets the filter, and that day arrives — usually in an internal admin tool, or a new endpoint, or a background job that was written by someone who did not know the convention. A hard boundary cannot be forgotten because there is no query that spans it.

The cost is that per-tenant analytics and cross-tenant search become genuinely impossible rather than merely discouraged. That is usually the right trade for anything customer-facing.

3. What you do about deletes

Deletes are the part every integration under-implements, because they are invisible when they go wrong.

When content is removed on your side, the document has to be removed from the index — otherwise your search keeps confidently answering from material you have unpublished, which in some domains is a compliance problem rather than a quality one.

Two patterns work. Fire a delete when your own delete happens, which is precise and misses anything that bypasses your normal path. Or reconcile periodically: list what the index holds, diff it against what should be there, delete the difference. The second is slower and catches everything, and most mature integrations end up doing both.

• Delete on your own delete event, for timeliness.
• Reconcile on a schedule, for correctness.
• Log the reconciliation diff. A diff that is consistently non-empty means your event path is dropping something.

4. What to do with conversation history

If your search is conversational, you have to decide who owns the transcript. There are two models and the difference has real consequences.

The stateless model costs you a little more integration work and gives you a much simpler answer to "what do you store about our users". If you are selling into anyone with a privacy review, that answer is worth the extra work.

Either way, passing prior turns improves results measurably — it is what lets "which of those?" resolve to something, both when extracting entities from the question and when phrasing the answer.

5. Whether you render their answer or write your own

Most retrieval services now return both ranked passages and a generated answer. You can use either.

Rendering theirs is faster to ship and gets you grounding and citation validation you did not implement. Writing your own gives you control of tone and format, and means you own the prompt when it needs tuning for your domain. The middle path — take their passages, write your own prompt — is common and reasonable, but be aware you are then re-implementing grounding, marker validation and the refusal path yourself. Those are the three things from the grounding guide, and skipping any of them is how a well-retrieved answer becomes a confident wrong one.

Sequencing the work

A rough order that avoids rework:

1. Pick the identifier scheme. Write it down somewhere your future self will find it.
2. Ingest one document by hand, with curl. Confirm you can search it.
3. Wire ingest into whatever already fires when content changes. Do not build a scheduler yet.
4. Add the delete path. Now, while the mapping is fresh, not later.
5. Put a search box in front of an internal user. This is where you find out whether your content contains the answers.
6. Add the reconciliation job once you have enough documents that drift is plausible.
7. Only then worry about tuning relevance.

What not to build yet

• A sync scheduler. Start with the event you already have. Add scheduling when you know what drifts.
• A relevance-tuning surface. You do not yet know which queries fail; tuning before you do is guessing.
• Your own caching layer. Search latency is dominated by answer generation, which is not cacheable in the way you are hoping.
• A feedback widget. Useful eventually, useless before you have traffic — read the search history instead. The questions with refusals attached are your content roadmap.