A Cache Breakpoint Lint Rule for Prompt Templates

The lint rule only works if the template itself knows where the cacheable region ends. Do not let the breakpoint live in application code, where a refactor can move it without touching the template. Put a marker in the file.

A minimal format: a line starting with @@CACHE_BREAKPOINT: name. Everything above the marker is the cacheable prefix. Everything below is the volatile suffix (the user's message, retrieved chunks, per-request timestamps). The renderer reads the marker, splits the file, and emits the API call with a cache_control block on the last prefix block.

@@BLOCK: tools
You have three tools: search, fetch_url, run_sql.
@@BLOCK: policy
You are a support analyst. Cite every claim. Refuse to speculate.
@@CACHE_BREAKPOINT: support_v3
@@BLOCK: request
User question: {{question}}

Three properties fall out of this format. The breakpoint has a name, so the test can report it. Blocks are named, so the test can say which one changed. The marker is plain text, so any editor, diff view and CI runner can read it without a parser dependency beyond your own code.

Name the breakpoint with a version or a purpose, not a number that changes when someone reorders lines. If you version it, the test's failure message will say "breakpoint support_v3 moved from line 12 to line 9", which is exactly the sentence a reviewer needs.

There is a second reason to make block boundaries explicit. The invalidation hierarchy is tools, then system, then messages, and a change at one level invalidates that level and everything after it (breakpoint placement walkthrough). If your template mixes a tool definition into a late block, one edit to the tool definition resets far more than the author expected. The block structure makes that blast radius visible before the edit lands.

The test is short. Parse each template, find the marker, hash the text before it, and compare against a stored value keyed by template path and breakpoint name. A mismatch fails and prints the diff location.

import hashlib, pathlib, json, re

MARKER = re.compile(r"^@@CACHE_BREAKPOINT:\s*(?P<name>[\w.-]+)\s*$", re.MULTILINE)
BLOCK = re.compile(r"^@@BLOCK:\s*(?P<name>[\w.-]+)\s*$", re.MULTILINE)
SNAPSHOT = pathlib.Path("tests/cache_prefix_snapshot.json")

def split_prefix(text):
    m = MARKER.search(text)
    assert m, "template is missing a CACHE_BREAKPOINT marker"
    return m.group("name"), text[: m.start()]

def digest(text):
    return hashlib.sha256(text.encode("utf-8")).hexdigest()

def blocks_in(prefix):
    return [m.group("name") for m in BLOCK.finditer(prefix)]

def test_cache_prefix_unchanged():
    stored = json.loads(SNAPSHOT.read_text())
    failures = []
    for path in sorted(pathlib.Path("prompts").glob("*.tmpl")):
        text = path.read_text(encoding="utf-8")
        name, prefix = split_prefix(text)
        key = f"{path}::{name}"
        actual = {"sha256": digest(prefix), "blocks": blocks_in(prefix)}
        expected = stored.get(key)
        if expected is None:
            failures.append(f"{path}: breakpoint '{name}' has no stored hash")
        elif expected != actual:
            failures.append(describe_drift(key, expected, actual))
    assert not failures, "\n".join(failures)

The test uses pytest's ordinary discovery conventions, so it runs wherever the rest of the suite runs (pytest good practices). The same command works in a GitLab or GitHub pipeline as a stage step (running pytest in CI).

The interesting part is describe_drift. Comparing the block name lists tells you whether the breakpoint moved or the content changed.

def describe_drift(key, expected, actual):
    exp, act = expected["blocks"], actual["blocks"]
    if exp != act:
        added = [b for b in act if b not in exp]
        removed = [b for b in exp if b not in act]
        return f"{key}: cache prefix block order changed. added={added} removed={removed}"
    return f"{key}: content changed inside cached prefix, blocks {exp} unchanged in order"

The first message says a block entered or left the cacheable region, which almost always means the breakpoint marker moved by accident. The second says someone edited a cached block in place, which is often intentional but needs review. Neither message requires you to reason about token counts or hashes.

The snapshot file is a normal JSON file in the repo. Reviewers see the hash change in the diff, next to the template change, in the same commit. That is the whole design: the hash does not prevent edits, it forces the edit and the acknowledgement to travel together.

The lint rule cannot tell you whether a given piece of text is stable. That decision is yours, and it is where most of the value sits. Use a stability test rather than an intuition test: if two consecutive requests in the same session would render different text for a block, it does not belong above the marker.

The ordering rule from the same body of work: stable content first, volatile content last, never change what came before (five ways to invalidate your prompt cache). A team that moved dynamic working memory out of the system prompt and into a trailing user message raised its cache hit rate from 7 percent to 84 percent and cut overall LLM cost by 59 percent (prompt caching guide). That is a single reordering, not a model change, and the lint rule would have caught the original layout on the first push.

For retrieval pipelines, chunk-level caching research makes the same point structurally: prefix reuse is safe by construction when prefixes match, but everything downstream of an unstable insert becomes fresh work (Grounded Cache Routing). RAGCache builds a knowledge tree of cached states and reports up to 4x lower time to first token than a baseline without it (RAGCache paper). Those gains evaporate if the injected document order changes per request, which is precisely what a per-template prefix hash will show you at review time.

Consider a support agent with a 12,000-token cacheable prefix and 500-token requests, so the prefix dominates. Reads bill at 0.1x base input and writes at 1.25x on the 5-minute tier, per the published multipliers (Anthropic caching pricing mechanics). Track a burst of 20 requests in five minutes, all sharing the prefix.

With a stable prefix: one write (12,000 tokens at 1.25x) plus 19 reads (12,000 tokens at 0.1x each). With a broken prefix: 20 writes and zero reads, because each request differs before the breakpoint.

That is roughly an 8x difference on input cost for the same 20 requests. At higher volume the ratio moves toward 12.5x, and the break-even is low: the 5-minute tier pays for itself after a single read, so almost any reuse makes caching correct and almost any silent miss makes it wrong (Anthropic caching pricing mechanics). Also note that a 12,000-token prefix clears the minimum cacheable size for every tier described there, while a shorter prefix may silently never cache at all. The lint rule protects you from the first failure mode; a token-count assertion against the documented minimum protects you from the second.

The latency half matters too. Caching reduces time to first token sharply while leaving generation speed unchanged, so a miss is a visible stall for a user waiting on a response (prompt caching explained). A cost-only framing understates the damage.

Wire the test as a normal unit test job. It reads files and hashes strings, so it runs in well under a second per template and needs no network, no API key and no model call. That means it can run on every pull request, including from forks where secrets are unavailable.

Two pieces of enforcement make the rule stick beyond CI. A pre-commit hook that runs the same test locally catches drift before a push, so the developer who knows why the prefix changed is the person who updates the snapshot. A CODEOWNERS rule on the snapshot file forces a prompt owner to approve hash changes, which turns the diff into a review checkpoint rather than a rubber stamp.

Keep the snapshot generation in the same script as the check. One command with a --update flag rewrites the hashes and prints a summary of what changed; the CI job calls it without the flag. Resist the temptation to auto-update on failure. The entire point is that a hash change is a decision someone made, not a side effect of a passing build.

Scope matters as the number of templates grows. Hash per template and per breakpoint, not one hash for the whole directory, so a failure names one file. If you support several providers, the same template can carry multiple breakpoints (one for an explicit cache_control provider, one for a automatic-prefix provider), and the snapshot key is the pair. If your prompt library starts to span many models with different caching rules, routing them through one interface keeps the test surface manageable: Quanta Inference exposes text, image, video, audio and embedding models through a single OpenAI-compatible key, which keeps the template and its breakpoint deterministic across providers (https://quantaintelligence.ai/inference).

A cache breakpoint test protects prefix reuse. It says nothing about correctness of reused answers, and confusing the two produces both false confidence and unnecessary fear.

Prefix and KV reuse preserves token outputs exactly when prefixes match, while output-level semantic caches do not carry that guarantee: a similar query can map to a cached answer whose evidence is stale or absent, and naive semantic caches have been shown to be adversarially exploitable (Grounded Cache Routing). The same paper reports an evidence-validated router driving the wrong-answer rate to zero on one benchmark family versus 15 to 35 percent under naive semantic caching, and to 1.5 percent versus 51.5 percent on a document-drift dataset. Those numbers are about answer reuse, not prefix reuse, and they are a reason to gate semantic caching behind evidence checks, not a reason to distrust your breakpoint.

Practically: your lint rule answers "did the cacheable prefix change?" Answer caching needs a different gate, keyed on source version and evidence overlap. Keep them in separate test files with separate failure messages so nobody tries to fix one by weakening the other.