Freshness SLOs: A Staleness Budget per Source

A staleness budget is one number per source: the maximum age, in seconds or minutes, that a consumer may be served. It is a business statement, not a technical one, and it almost never comes from engineering. A price feed, a support-article body, a shipping address and a product catalog all deserve different numbers, and treating them as one fleet-wide cache TTL is how silent staleness leaks into decisions.

A practical way to set the number is to ask what breaks when the data is older than X. If a stale value causes a wrong charge, the budget is minutes at most. If it causes a mildly wrong recommendation, hours may be fine. Record the answer, the owner and the date next to the number so the next person can revisit it with context.

A checklist for choosing the budget:

Two rules keep the table honest. First, the budget is an upper bound on age, not an average, so the mechanism must guarantee it in the worst case, not on a good day. Second, every source gets its own row; copying a neighbor's number is guessing.

The SLO is the budget expressed as a measurable objective over a window. Choose one of two indicator styles and stay consistent:

• Time-based: the fraction of minutes in the window during which every record in the source was within its staleness budget. A 99.5% target over 30 days allows about 216 minutes of violation (0.5% of 43,200 minutes).
• Read-based: the fraction of reads served with data inside the budget. A 99% target over a million reads allows 10,000 stale reads.

Time-based indicators suit sources that feed dashboards and reports, where a silent hour is an hour of wrong decisions. Read-based indicators suit high-traffic answer paths, where a stale read is a discrete event you can count and attribute. Both are legitimate; mixing them in one report is not, because they answer different questions.

The error budget is the complement: 100% minus the objective. A 99.5% time-based SLO on a 30-day window gives 216 minutes to spend. That number is the whole point. It converts an argument about whether freshness is good enough into arithmetic: you are inside budget, so ship the risky migration; you are outside, so freeze changes to that source's pipeline until it is replenished. Error budget policies formalize exactly this trade, so that when the budget is exhausted agreed actions take effect rather than a debate (Netdata, designing error budget policies).

A worked example makes the arithmetic concrete. A source with a five-minute staleness budget has 43,200 five-minute slots in a 30-day window. At 99.5%, the SLO allows 216 violating minutes. If last month's incident burned 180 minutes, the source has 36 minutes left. A planned migration that will drop the change stream for 20 minutes fits; one expected to take an hour does not, and should either wait for the monthly reset or be funded with a faster recovery path.

The most common measurement bug is to timestamp the fetch. If a job runs every minute and the source last changed six hours ago, most implementations record an age of under a minute and the SLO looks perfect while consumers read a six-hour-old value. Age must be measured from the last confirmed change at the source, or from a source-maintained updated-at column you trust.

That distinction is why change data capture exists. Polling asks what the data looks like now, and it misses updates when timestamps overlap, duplicates on retry, and deletes become invisible unless handled manually, with lag equal to the poll interval (Understanding Change Data Capture with Debezium). Log-based capture reads the database's own transaction log and emits an event for every insert, update and delete, so the pipeline knows when the source actually moved. Debezium uses this approach and publishes changes to Kafka, where downstream consumers can react (Instaclustr, CDC with Kafka and Debezium).

The other side of the coin is invalidation, and it is genuinely hard. A cache that is not told about a change keeps serving the old value indefinitely. Debezium's own write-up on automating invalidation with CDC frames the problem plainly: when changes bypass the ORM, nothing tells the cache that its entry is stale, so you must invalidate explicitly or keep serving outdated data (Debezium, automating cache invalidation with CDC). Their Hibernate extension scans the model at build time, registers handlers for the relevant tables, and evicts cache regions when it sees an update or delete (Debezium Hibernate cache docs).

For the LLM layer, the same lesson applies to prompt caching. Provider caches key on stable prefixes, and reordering a system prompt or injecting a timestamp near the top invalidates every entry, which can silently halve a hit rate and surface later as a cost spike (Metabase, cache hit rate). A new prompt template is a data change to a cached source, and should be treated as one: versioned, announced and checked against the freshness SLO of the configuration it feeds.

An alert that fires when the SLO is breached is an alert that fires after your users already saw stale data. Alert on the rate at which the budget is being consumed instead, so a source on track to exhaust its budget in a day pages someone while there is still room to recover.

The standard pattern is multi-window, multi-burn-rate alerting: a short window catches fast burns, a long window confirms the trend and suppresses noise, and the combination avoids both flapping and slow detection. The Google SRE Workbook's chapter on implementing SLOs is the usual reference for why the objective and its budget should drive the alert rather than raw thresholds (Google SRE Workbook), and the same reasoning transfers to data quality targets (Muness Castle).

For a freshness SLO, translate burn rate into age. If a source's budget is five minutes and it has been stuck at nine minutes for an hour, that is a burn rate of 100% of the budget every hour, and it deserves a page, not a ticket. Two thresholds cover most sources:

• Fast burn: the source has been over budget for several consecutive checks and will exhaust its monthly budget within a day at this rate. Page the owning team.
• Slow burn: the source is over budget intermittently, and the monthly budget will be gone by the end of the week. Open a ticket with the offending check and window.

Treat any source whose age has stopped increasing as suspect. A silent source is not a fresh source, and a pipeline that has been quietly failing to deliver events will pass an age-from-fetch check forever. That is why the measurement must come from the source's own change record.

The final step is the one most systems skip: enforce the contract when answering. A freshness SLO that lives only in a dashboard is a report; one that is checked on the read path is a contract.

The pattern is a gate, applied before the value is returned:

def read_with_freshness(source, key, now):
    record = store.get(source, key)
    if record is None:
        return escalate(source, key, reason="missing")
    age = now - record.source_changed_at  # not the fetch time
    budget = BUDGETS[source]              # e.g. 300 seconds
    if age <= budget:
        return record.value
    if budget_state(source, now).remaining > 0:
        # Budget is blown for this record but the source still has room.
        return escalate(source, key, reason="stale", age=age)
    return refuse(source, reason="freshness budget exhausted")

The escalate path is the design decision. It might refresh synchronously from a slower but authoritative path, call a human, or return the value with an explicit freshness flag and a warning. The important part is that it does not look like a normal answer. Serving a stale value silently is the failure mode the entire SLO exists to prevent, and an explicit refusal is a much better outcome than a plausible wrong number.

For AI systems, the gate belongs in front of the context assembly, not after generation. If a retrieval source is over budget, the model should be told, or the request should route to a slower path that re-reads the source. Semantic caches deserve particular care here, because an embedding-close match is not a meaning-equal one, and the wrong cached answer arrives with a 200 status (TrueFoundry, semantic caching). A semantic cache entry should carry the freshness budget of the sources it was built from, and a gate should reject it when those sources have moved.

If you are building this and would rather not stand up the inference plumbing yourself, Quanta Inference puts text, image, video, audio and embedding models behind one OpenAI-compatible key, which keeps the gate logic in your code and the model calls in one place.

Only now does the mechanism question get a clean answer. Compare each option's worst-case lag against the staleness budget, and pick the cheapest option that fits with margin.

Webhooks are the fastest option when they work, but they fail in ways that look like success. A forged request can inject bad data, and a replayed one can resurrect a deleted record, which is why signature verification, timestamp validation and idempotent processing belong in every receiver (Hookdeck, webhook security). Polling is simpler and usually worse for freshness, since the change data is out of date for as long as the poll interval and deletes tend to disappear (Instaclustr, CDC). Log-based capture is the strongest option for sources that change often and matter, at the cost of running streaming infrastructure.

One decision rule settles most debates: if the budget is under a minute, use log-based capture or a verified webhook; if the budget is hours, a TTL with an age-from-source check is enough; if you cannot state the worst-case lag of your mechanism, you do not have a freshness SLO, you have a hope.