What to Store with Each Passage So Retrieval Can Be Audited

The minimal auditable passage record has five fields. Each is optional in some systems and mandatory here, because dropping any one of them removes a distinct diagnostic path.

A sixth field, a passage identifier, is assumed. It is the primary key that lets you look up a record at all.

The important design decision is that source_id and ingest_ts are immutable for a given passage record. If the source document changes, you write a new record with a new doc_version and a new ingest_ts, then delete or tombstone the old one. Mutation destroys the audit trail: a record whose doc_version and ingest_ts were overwritten can no longer tell you what was indexed when.

A minimal record looks like this in most vector stores, where the metadata payload carries everything except the embedding:

{
  "id": "psg_9f31c2",
  "source_id": "hr-policy-remote-work",
  "doc_version": "rev-42",
  "section": "Section 4.2 Eligibility",
  "acl": ["group:all-employees"],
  "ingest_ts": "2026-03-02T14:11:07Z"
}

Two conventions save trouble later. Store timestamps in UTC with an explicit zone, so comparing an ingest_ts against a query log is a subtraction rather than time-zone archaeology. Store ACL as a list of stable group identifiers, never display names, because names get edited and identifiers do not.

source_id is the stable key for the source document. Take it from the source system where one exists: a document management ID, a path in a version-controlled repository, a ticket number. Where the source has no identifier, derive one from a canonical URL or path at ingest and keep it in a mapping table. The failure to avoid is deriving source_id from the document title, because titles get edited and the identifier silently changes underneath you.

doc_version answers "which revision of the source does this passage represent?" The simplest reliable form is a content hash of the parsed source, which changes exactly when content changes and asks nothing of the source system. Where the source carries its own revision numbers, store both. doc_version is what lets you state, with confidence, that the passage retrieved for a query came from the revision of the policy in force at the time, or that it did not.

section is the locator inside the document. For structured documents, use the heading hierarchy ("Section 4.2 Eligibility"). For paginated documents, use a page range, which is also what page-level chunking preserves: page-level chunking performed well in published benchmarks for paginated documents specifically Firecrawl. The section field changes whenever the document is re-chunked, so treat it as a property of the passage rather than of the document.

ACL is the access scope required to see the passage. It is populated by copying permissions from the source system at ingest and kept current by an ACL sync job. This is the field most often lost in flight: source permissions can be dropped when a document is split into chunks and embeddings, and authorization must stop unauthorized text from entering the model context rather than relying on the model to ignore it OneSourceCloud. A passage without a populated ACL is either invisible to everyone or, worse, visible to everyone. Neither is defensible, so make ingest fail closed: a missing ACL blocks the write.

ingest_ts records when the passage entered the index and never changes. It is the field that separates a stale index from a weak retriever, and the one teams most often skip, because it feels like infrastructure trivia until the first answer that cites a superseded policy.

Suppose a user asks a question and the system answers from an old policy, or returns nothing useful. The audit is a short sequence of checks, and each check leans on one field.

Step 1: confirm the passage existed. Look up the cited passage by identifier and read its record. If the user insists the right answer exists but the system cited nothing, search the index directly by source_id and section. A miss here is a coverage failure in the data layer, not a retrieval failure, and no retriever tuning will surface text that was never indexed Olostep.

Step 2: trace the passage to its source and version. Fetch the source document at doc_version and confirm the stored passage text matches. A mismatch means the parser or chunker produced text that differs from the source, a data-quality problem that will recur until the parser is fixed. A match means the index faithfully represents a revision of the document, and the question becomes whether it was the right revision.

Step 3: check the ACL. Take the ACL from the passage record and evaluate it against the groups the querying user belonged to at query time. If the user's groups matched no ACL entry, the passage was excluded by design and the answer was right to omit it. This check stops you from launching an expensive retrieval investigation when the real cause is that a permissions sync removed a user from a group, or that a document was classified more tightly than its readers expected. Security audit practice tests these cases explicitly:

That table comes from a security audit framework that treats retrieval quality as a control in its own right, on the grounds that poor retrieval becomes a security problem when it cites obsolete policy or omits a critical warning OneSourceCloud.

Step 4: compare ingest_ts with the query time. If the query was written before the passage was ingested, the index was not stale; the question predates the content and the answer is correct. If the query came after ingest_ts and a newer doc_version of the same source is already in the index, the system may have retrieved an older passage because both revisions ranked close together. This is where version-aware filtering pays off: when sources revise often, filter to the current doc_version per source_id at query time, or boost current versions in ranking.

A worked example makes the arithmetic plain. The user asked on 3 June. The retrieved passage has ingest_ts of 14 May, doc_version rev-41, and ACL of group:all-employees. The source's current revision, rev-42, was ingested on 28 May. The user belongs to all-employees. The passage was present, authorized, and indexed before the query, but it was not the current revision. The diagnosis is a versioning problem, not a permissions or freshness problem, and the fix is to prefer current revisions in ranking rather than to re-ingest anything.

Metadata lets you audit individual failures. It also lets you measure the pipeline, provided retrieval and generation scores stay separate. Retrieval metrics ask whether the right passages ranked near the top; generation metrics ask whether the answer was supported and correct; measuring them apart turns a vague "quality is low" into a diagnosis that names a component Braintrust.

The two classic retrieval measures are precision and recall. Precision is the fraction of retrieved instances that are relevant; recall is the fraction of relevant instances that were retrieved Wikipedia. In ranking systems both are computed at a cutoff K: precision at K divides the relevant items in the top K by K, and recall at K divides them by the total number of relevant items Evidently AI. A worked calculation shows the mechanics: with 80 relevant records in a database, a search returning 60 records of which 45 are relevant achieves recall of 45/80, or 56 percent, and precision of 45/60, or 75 percent Creighton.

The catch is that recall requires knowing how many relevant records exist, which requires labels, and the labels must point at a specific document revision. A golden question that says "the remote work policy" is not a label; one that says "the remote work policy, rev-42, section 4.2" is. When reference answers expire because source pages change, questions need re-checking on a schedule and whenever a monitored source changes, and storing the source identifier with every golden question is what maps a changed page back to the questions needing review Olostep.

This is where the schema earns its keep twice. In production, source_id and doc_version let you trace why a specific query failed. In evaluation, the same fields let you catch drift: a rising share of passages whose doc_version is not current is a versioning problem you can see before anyone complains. Track three numbers on a dashboard: the share of indexed passages older than your freshness window, the share of sources with more than one live doc_version, and the share of golden questions whose reference source has changed since labeling.

One decision rule, because the precision-recall tradeoff is real and teams often tune it blindly. Precision and recall pull in opposite directions, and which matters depends on the application: a system that must find everything on a topic needs strategies that favor recall and tolerate noise, while a system that must return a few good results favors precision Creighton. For a compliance assistant, a missed current policy passage is a recall failure with legal consequences, so tune toward recall. For a customer-facing answer box, noise costs trust, so tune toward precision. The schema does not change the tradeoff, but it tells you which side you are failing on.

Vocabulary matters too. Stories about search quality are often stories about synonyms, and precision problems commonly come from the query and the index using different words for the same thing. Expanding queries with synonyms and related terms raises recall but lowers precision, because synonyms are rarely exact Creighton. When you audit a "nothing found" complaint, check whether query and passage share vocabulary before rebuilding anything, since the fix is sometimes query expansion rather than a change to the corpus.

The section field is the one tied to chunking, and chunking is not a neutral preprocessing step. A systematic study of fixed-size chunking across datasets found that chunk size materially affects retrieval: smaller chunks of 64 to 128 tokens worked best for datasets with concise, fact-based answers, while larger chunks of 512 to 1,024 tokens improved retrieval where broader context was needed, and different embedding models showed different sensitivities to chunk size arXiv.

The implication for the schema is that section should identify the unit a reader would use to find the passage, not merely a token offset. A 400-token passage that spans the end of one section and the start of another should record both, or be split. If your chunking strategy changes, every section value in the index changes with it, which is why section churn is tracked separately from doc_version: re-chunking is not a new document revision, and conflating the two makes audit logs unreadable.

Retrieval quality also shifts with how the pipeline handles chunks. Chunk boundaries, embedding choices, lexical search, metadata filters, rerankers and corpus freshness can all change the result for the same model and prompt Online Inference. The five-field schema pins down three of those variables (freshness, revision and access scope), so when you compare two configurations the difference is the thing you changed rather than the corpus drifting underneath you.

For teams on managed knowledge bases the same principle applies one level down. Amazon Bedrock Knowledge Bases, for example, manages ingestion, indexing and retrieval over your data sources, with options to use your own models for embedding and ranking AWS. Even with managed retrieval, store the five fields in the chunk payload wherever custom metadata is allowed, because the audit questions are yours to answer, not the platform's.

When the retrieval layer is stable and the remaining sprawl is model calls across embedding, reranking and generation, Quanta Inference puts text, image, video, audio and embedding models behind one OpenAI-compatible API with prepaid pricing in US dollars: https://quantaintelligence.ai/inference.