How to reduce hallucination with enforced citations

Asking a model to cite its sources reduces hallucination somewhat. Checking the citations afterwards reduces a specific and damaging subclass of it almost entirely, and costs about fifteen lines of code. This guide is about the gap between those two things.

Three failures that all look like one

"Hallucination" covers several distinct problems with different fixes. Separating them is most of the work.

The middle row is the one to fix first. It is the cheapest, and it does the most for trust: a reader who follows a footnote and finds nothing stops believing every other footnote on the page.

Marker validation, concretely

Number your sources in the prompt, require numeric markers in the output, then check what came back against what you supplied.

function validateCitations(answer, sources) {
  const supplied = new Set(sources.map((_, i) => i + 1));
  const cited = [...answer.matchAll(/\[(\d+)\]/g)].map((m) => Number(m[1]));

  const invented = [...new Set(cited.filter((n) => !supplied.has(n)))];
  const used = [...new Set(cited.filter((n) => supplied.has(n)))];

  let text = answer;
  for (const n of invented) text = text.replaceAll(`[${n}]`, "");

  return {
    text: text.replace(/\s+([.,;])/g, "$1").trim(),
    citations: used.map((n) => ({ n, documentId: sources[n - 1].documentId })),
    inventedCount: invented.length,
  };
}

Two details worth keeping. Drop the marker rather than the answer — the prose attached to a mis-numbered marker is usually correct, and rejecting the whole response is a worse outcome than removing a footnote. And tidy the whitespace afterwards, or you leave "as documented [3] ." in your output.

Return the citation list, not just the text

Once markers are validated you know exactly which sources were used. Return that as structured data alongside the prose rather than making the caller re-parse your own output.

{
  "text": "Cohorts completing onboarding within seven days
           show lower churn at ninety days [1], and the
           support record attributes it to activation [3].",
  "citations": [
    { "n": 1, "documentId": "doc_01j...", "externalId": "retention-q3" },
    { "n": 3, "documentId": "doc_01k...", "externalId": "support-2026" }
  ]
}

Including your caller's own identifier for the document, not just yours, is the difference between a citation they can render as a link into their product and one they have to look up.

The invented-marker rate is a retrieval metric

This is the most useful thing to come out of validation, and it is easy to miss.

Count invented markers over time. When the rate rises, the usual cause is not that the model got worse — it is that retrieval is returning too little, so the model is reaching past the supplied sources to complete an answer. It is a leading indicator for a retrieval regression, and it costs one counter.

Do not let derived context be citable

If you supply anything beyond raw passages — a summary, a knowledge-graph fact, a computed relationship — it must be marked non-citable in the prompt and excluded from the valid marker range.

A derived fact is your reading of the sources, not a sentence any source contains. Cite it and a reader who follows the link lands on a document that does not say what was cited — the specific failure that marker validation exists to prevent, reintroduced through the back door.

What about entailment checking?

The remaining failure — a marker that resolves but attaches to the wrong claim — needs an entailment check: for each sentence, does the cited passage actually support it? That is a model call per sentence.

It is usually not worth it, for two reasons. The cost scales with answer length on every request, and the check itself is a model judgement that can be wrong in both directions — you can reject correct citations and accept incorrect ones. If you do it, do it as an offline audit over sampled traffic rather than inline, and use it to tune retrieval rather than to gate responses.

The order to do this in

1. Number sources in the prompt and require numeric markers. Free.
2. Validate markers and drop the invented ones. Fifteen lines.
3. Return citations as structured data with the caller's identifiers. Small.
4. Count invented markers and refusals, and watch both. One counter each.
5. Add refusal controls to your evaluation set. A morning of work, and the only thing that catches "answers more" masquerading as "grounds better".
6. Consider offline entailment auditing. Optional, and last.

The first four take an afternoon and remove the failure mode that costs you the most credibility. The fifth is what stops you fooling yourself about whether any of it worked.