How to Keep a Support Retrieval Index Fresh as Articles Change

Three signals cover almost every edit:

• Content hash. Compute a hash over the normalized article body (strip navigation, comments, tracking parameters, and whitespace runs first). A changed hash means the text changed; an unchanged hash means skip.
• Last-modified time. Use the source system's own timestamp as a cheap pre-filter and as an audit record. Timestamps lie (bulk exports touch everything), so never trust them alone.
• Diff. When the hash changes, fetch the new text and diff it against the stored version. The diff is what lets you classify the edit rather than guess.

A practical record per article looks like this:

CREATE TABLE article_index_state (
  article_id        TEXT PRIMARY KEY,
  source_url        TEXT NOT NULL,
  content_hash      TEXT NOT NULL,      -- sha256 of normalized body
  source_modified   TIMESTAMPTZ,        -- from the source system
  indexed_at        TIMESTAMPTZ,        -- when chunks were last written
  index_version     TEXT,               -- embedding model + chunker version
  status            TEXT,               -- active | redirected | retired
  canonical_id      TEXT                -- NULL unless this is a duplicate
);

The index_version column matters as much as the hash. Content can be unchanged while the embedding model or chunking configuration changes underneath it, which is a separate kind of staleness you must handle deliberately (see the section on embedding rot).

For sources that push events (webhooks from your help center, wiki or repository), process the event directly. For sources that do not, poll on a per-source cadence. Different source types belong on different clocks: ticket and wiki content changes far faster than a documentation site that redeploys nightly kapa.ai's source refresh intervals.

Not every hash change deserves a reindex. A typo fix that alters a single character will trigger your diff, and re-embedding the article costs money and briefly risks serving a half-written index. Classify first.

A simple decision rule that holds up in practice:

1. If the diff touches no sentence that answers a user question (numbers, conditions, steps, policy language, product names), mark it cosmetic and let the nightly batch handle it.
2. If the diff changes any answer-bearing sentence, reindex that article within minutes.
3. If the diff changes the article title, section headings or structure, reindex immediately and re-run your evaluation set, because chunk boundaries moved.

You can automate step 1 reasonably well by extracting answer-bearing spans: sentences containing digits, currency symbols, dates, modal verbs ("must", "cannot", "only if"), or imperative steps. Flag any diff that intersects one. When a model does the classification, schema-constrained output is far safer than free text, and you should validate the result before acting on it MLflow on structured output for LLM pipelines.

A worked example makes the trade-off concrete. Suppose your support corpus has 4,000 articles averaging five chunks each, 20,000 chunks total. A full reindex embeds 20,000 chunks; at an embedding price of a fraction of a cent per thousand tokens, the dollar cost is small, but the pipeline time and the risk window are not. If a typical weekday produces 60 article edits, of which 15 are substantive and 45 cosmetic, incremental reindexing touches 75 chunks instead of 20,000. That is roughly 0.4 percent of the work, which is why you can afford a minutes-level target for the substantive ones.

Incremental (delta) updates re-process only what changed, while full reindexing rebuilds everything; the former keeps a knowledge base current far more cheaply and is the default at scale kapa.ai. Most large deployments combine the two: frequent incremental updates plus an occasional full rebuild, for example after a chunking change.

Two implementation details separate a working pipeline from a fragile one.

First, delete before insert. Reindexing an article means removing its previous chunks (keyed by article_id) and writing the new set in the same operation, so no window exists where both old and new chunks are retrievable. If your vector store supports it, write the new chunks under a new index_version and flip a pointer atomically.

Second, keep chunk IDs derived from content, not from position. A chunk ID built from article_id plus the hash of the chunk text means an unchanged paragraph keeps its identity across edits. That protects caches, saved citations and any feedback attached to specific chunks.

Chunking choices interact with freshness. Fixed-size splitting is simple and predictable but cuts mid-sentence; recursive or sentence-aware splitting preserves natural boundaries and suits knowledge base prose; semantic and structure-aware splitting cost more at index time but produce more coherent chunks SurrealDB's comparison of chunking strategies. For support articles, a structure-aware approach that splits on headings, with a sentence-aware fallback inside long sections, is a sensible default. Managed services expose the same trade-off: Amazon Bedrock knowledge bases offer standard, hierarchical, semantic and multimodal chunking modes AWS documentation on content chunking.

Whichever you choose, version it. A change to chunk size or overlap is a reindex event for the whole corpus, not a per-article one.

Retiring an article is the highest-risk operation in a support index, because the article usually still exists in someone's memory and often still ranks in web search. Do three things, in this order:

1. Remove the chunks. Delete every chunk for the retired article ID and record status = 'retired' with a timestamp. Leaving them in place is the "ghost answer" failure: users get instructions for a product or plan that no longer exists.
2. Add a redirect. Map the old URL to the replacement article. If your retrieval layer stores source URLs, rewrite the stored URL to the successor so citations point somewhere real, and keep a redirect record so a user who followed an old link lands correctly.
3. Invalidate cached answers. If you cache generated answers keyed by query, clear entries whose citations include the retired article. Otherwise the stale answer survives the fix.

For near-duplicate articles, deduplicate at two levels. At the source level, cluster articles by embedding similarity on the full document and by title similarity, then pick a canonical article per cluster and redirect the rest. At the chunk level, keep one chunk per duplicate cluster rather than letting five near-identical chunks compete for the same query. Duplicate vectors from repeated content poison the index and degrade retrieval, and treating vectors as duplicates when cosine similarity exceeds a threshold is the practical way to catch them considerations for data processing in RAG applications.

The decision rule for which article is canonical: the one with the most recent substantive edit, the highest internal traffic, and the fewest unresolved contradictions against policy. If two articles disagree on a number, that is not a duplicate problem, it is a content bug. Route it to the owner before either is canonical.

Reindexing is necessary but not sufficient. Add three ongoing checks.

Contradiction sampling. Periodically select pairs of chunks that answer the same query and ask a model whether they conflict. When a newly indexed chunk contradicts an older one, one of them is stale even though both hashed as fresh. This catches the case where two articles were never reconciled.

Citation currency. Log which chunks were cited in production answers and when each was last verified. Any chunk cited repeatedly but not re-verified in a long period is a staleness candidate, regardless of its hash.

Retrieval evaluation on edit. Keep a small set of real support questions with known correct source articles. After any cluster of substantive edits, measure recall@k on that set; if the right article stopped being retrieved, the edit likely broke chunk boundaries or heading text. Retrieval quality and generation quality are separate problems and should be evaluated separately Evidently AI's guide to RAG evaluation. If you build a judgment-into-classification step for these checks, structure the prompt so the model applies explicit definitions and returns a fixed format, as fraud classification practice shows; well-grounded prompts produce more consistent classifications than open-ended ones the FedPayments Improvement blog on guiding LLMs.

A freshness dashboard that ties these together might show: articles with a hash change older than your latency target (should be zero), retired articles still present in the index (should be zero), duplicate clusters without a canonical article (should be zero), and recall@k on the evaluation set relative to its baseline.

Use this checklist to convert the rules above into a schedule.

The 10 to 15 percent churn threshold for re-embedding comes from the same sync discussion, which frames it as the point at which "embedding rot" warrants a fresh embedding pass kapa.ai.

Rough local models are fine for the model side of this pipeline, but you will still want consistent access to embedding models and any judgment models across sources. Quanta Inference puts text, image, video, audio and embedding models behind one API key, which removes a per-model integration from the refresh job.

When this runs well, the reader-facing effect is simple: an answer about a limit or a step reflects the article as it exists today, and a retired article never surfaces. When it runs badly, the model is confident and wrong, and nothing in the interface warns anyone. The pipeline is the warning system.