A Freshness Gate Before the Prompt: Fields and Refusals

Every chunk entering the gate needs four fields. Three live on the document, one lives in configuration.

Three rules make these fields usable in practice.

First, doc_ts must be the time the content changed, not the time the crawler last touched the document. A nightly re-crawl that re-writes every row with a new updated_at destroys the signal entirely. This is the same trap that change data capture was invented to solve: polling updated_at > last_run produces missed updates when timestamps overlap or jobs retry, and deletes vanish unless handled separately (the Debezium CDC explainer). If you control ingestion, capture the source's own modification signal and fall back to first-seen time, never to crawl time.

Second, if doc_ts is missing, do not default it to now. Defaulting to now silently converts every undated document into a fresh one, which is the failure mode you were trying to catch. Treat a missing timestamp as a distinct state that the volatility class resolves.

Third, source_id must be a closed set. Free-text source names drift, and a drifted name falls through to whatever your default threshold is. Validate source identifiers against the registry at ingest time and reject unknown ones.

A single global TTL is the usual first attempt and it fails in both directions. Set it to seven days and your architecture notes get refused for no reason; set it to a year and last quarter's pricing page sails through. The fix is a registry keyed by source, with the threshold chosen by how fast the underlying facts actually change.

The volatility class is the decision rule. Use it to pick the threshold, then override per source where you have evidence.

Two refinements turn this from a table into a system.

Refinement one: derive thresholds from observed change frequency rather than guessing. If you log every doc_ts per source, you can compute how often content actually changes and set the TTL at a multiple of that interval. A source that changes roughly monthly does not need a weekly TTL; a source that changes daily cannot survive one.

Refinement two: treat the TTL as a service level objective for data, not a hard wall. The SRE literature on SLOs is explicit that 100% is neither achievable nor desirable, and that an error budget tells you when to act (the SRE Workbook chapter on implementing SLOs). Applied to freshness, that means you define a target such as "at least 99% of answers are grounded only in documents within their source TTL," track the actual percentage of stale retrievals as the indicator, and use the gap to decide whether to invest in faster ingestion. The argument for extending SLO thinking to data quality is made directly in Muness Castle's write-up, which frames data quality the same way: a target, a measured indicator, and a budget for the remainder.

The function returns one of three outcomes, each with enough detail for the caller to act.

from dataclasses import dataclass, field
from datetime import datetime, timedelta
from enum import Enum
from typing import Literal

class Action(str, Enum):
    PROCEED = "proceed"
    REFUSE = "refuse"
    ESCALATE = "escalate"

@dataclass(frozen=True)
class GateDecision:
    action: Action
    reason: Literal[
        "fresh",
        "stale",
        "missing_timestamp",
        "unknown_source",
        "no_ttl",
    ]
    source_id: str
    doc_ts: datetime | None
    age: timedelta | None
    ttl: timedelta | None
    volatility: str | None
    chunk_ids: tuple[str, ...] = field(default_factory=tuple)

REFUSE means the caller must not send this chunk to the model. Assemble a refusal message instead: name the source, state what it would have answered, and offer a re-fetch or a link. ESCALATE means the chunk is doubtful but not disqualified; route it to a human queue or a higher-effort retrieval path. PROCEED means use it, and attach the reason string so that downstream evaluation can separate "fresh" from "undated but tolerated."

The reason field is the part teams skip and later regret. Without it, every refusal looks identical in logs and you cannot tell an ingestion outage from a misconfigured threshold. With it, a spike in unknown_source is a registry bug, a spike in missing_timestamp is a parser bug, and a spike in stale is a genuine pipeline lag. These are three different on-call runbooks.

One more design note: return the decision for the set of chunks, not just the first failure. If five chunks come back and four are fresh, refusing the whole answer is wrong; the caller should drop the stale one and proceed with the rest, or escalate if the stale chunk carried the specific fact asked about. That second judgment needs a query-level signal, so keep the gate per chunk and let a thin wrapper aggregate.

The gate composes with whatever retrieval stack you already run. The important integration points are three: where the timestamp comes from, where the registry is loaded, and what the caller does with the decision.

REGISTRY = {
    "pricing-page":   {"ttl": timedelta(hours=6),  "volatility": "high"},
    "api-reference":  {"ttl": timedelta(days=14),  "volatility": "medium"},
    "architecture-adr": {"ttl": None,              "volatility": "static"},
}

def gate(chunk, now, registry=REGISTRY):
    src = chunk.source_id
    if src not in registry:
        return GateDecision(Action.ESCALATE, "unknown_source", src,
                            None, None, None, None, (chunk.id,))
    cfg = registry[src]
    ttl, vol = cfg["ttl"], cfg["volatility"]

    if chunk.doc_ts is None:
        action = Action.REFUSE if vol == "high" else Action.ESCALATE
        return GateDecision(action, "missing_timestamp", src,
                            None, None, ttl, vol, (chunk.id,))

    age = now - chunk.doc_ts
    if ttl is None:
        return GateDecision(Action.PROCEED, "fresh", src,
                            chunk.doc_ts, age, None, vol, (chunk.id,))
    if age <= ttl:
        return GateDecision(Action.PROCEED, "fresh", src,
                            chunk.doc_ts, age, ttl, vol, (chunk.id,))
    action = Action.REFUSE if vol == "high" else Action.ESCALATE
    return GateDecision(action, "stale", src,
                        chunk.doc_ts, age, ttl, vol, (chunk.id,))

A worked example makes the thresholds concrete. A support assistant receives a question about a plan's overage rate. Retrieval returns three chunks: a pricing page dated 11 days ago with a six-hour TTL, an API reference dated 9 days ago with a fourteen-day TTL, and an architecture decision record with no TTL. The gate refuses the pricing chunk as stale, proceeds with the API reference as fresh, and proceeds with the ADR under the static class. The answer is assembled from two chunks, and the refusal on the pricing chunk triggers a re-fetch from the source of record. Without the gate, all three enter the prompt and the model may quote the older rate.

That ordering also saves money. Prompt caching gives large savings on repeated prefixes, with Anthropic's documentation pricing cache reads well below base input for the models it lists, and OpenAI applying a discount to cached tokens (the caching infrastructure overview). But a cached prefix built from a document you should never have sent is a cached mistake. Gate first, then cache.

TTL is a timer; it is not a notification. Between the moment a source changes and the moment its TTL expires, your gate will wave through content it would otherwise reject. For high-volatility sources, close that gap with change data capture rather than a shorter TTL.

CDC reads the database's own transaction log (the write-ahead log in PostgreSQL, the binlog in MySQL) instead of polling for changed rows, which eliminates the missed-update and retry-duplicate problems of the polling pattern (the Debezium CDC explainer). The canonical application is cache invalidation: Debezium's own walkthrough shows a Hibernate second-level cache going stale when records are modified outside the ORM, and CDC evicting the affected entries in near-real time (the Debezium cache invalidation post). The same mechanism applies here: a change event on a pricing table should rewrite doc_ts on the indexed chunk immediately, not at the next crawl.

If you run a Quarkus stack, the same team ships an extension that scans the JPA model at build time, registers handlers for the matching tables, and evicts cache regions on update or delete operations (the Debezium Hibernate cache extension docs). The pattern generalizes: subscribe to the write path, update the timestamp, and let the gate's existing thresholds do the rest. For sources you cannot instrument, the honest answer is a short TTL plus an escalation path, not a longer TTL plus hope.

There is a second invalidation gap worth naming: the gate trusts doc_ts. If ingestion writes the crawl time instead of the source time, every document looks fresh forever. Audit for this before you tune thresholds. The cheap test is to plot the distribution of doc_ts per source; if it clusters at exactly your crawl schedule, you are recording the wrong thing.

A gate that refuses too often is as damaging as one that never refuses. Take the SLO approach seriously: define the indicator, measure it, and let the budget tell you when to change a threshold.

Three metrics cover most of it. Refusal rate per source, which should be near zero for static sources and nonzero for high-volatility ones during ingestion lag. Escalation resolution time, since escalations that sit unanswered are really refusals with extra steps. And a sampled accuracy check: pull some refused chunks, verify they were genuinely stale, and pull some proceeded chunks and verify the opposite. The precision/recall tradeoff is the same shape as the one that governs similarity thresholds in semantic caching, where a threshold set too low serves wrong answers and one set too high collapses the hit rate, and the advice there is to tune it per route rather than globally (the semantic caching overview). Freshness thresholds are the same knob with a different label.

Watch one specific failure mode: a refusal that repeats. If the same source refuses every query for a week, the gate is not protecting the user, it is blocking an entire topic. That is an ingestion incident, and it should page someone. Treat sustained refusal as an availability signal rather than a quality signal.

Once the function exists, extending it is cheap. Per-tenant thresholds, query-intent overrides ("this user asked for historical pricing, do not refuse old documents"), and parallel gates for conflicting sources all fit inside the same return object. That extensibility is the whole argument for making the gate a function: policy ideas are hard to test, and functions are not.

If the retrieval side of your system is where the work keeps landing, Quanta Inference puts text, embedding, and other model types behind one OpenAI-compatible key, which keeps the gate's dependencies simple while you iterate on thresholds.