Injecting Dynamic Data into a System Prompt Safely

Sort every block in the prompt by how often it changes, measured across all requests, not across one conversation. The table below is a working checklist for a support or research assistant; adapt the rows, keep the sort.

Two rows deserve comment. Tool definitions sit high because they are usually static and often bulky, so a stable block of schemas pays for itself many times over. Retrieved documents sit low because their content is drawn from a corpus that changes continuously; a retrieval index that returns different chunks for similar queries will break prefix reuse no matter how you format them Weaviate on context engineering. Anthropic's guidance on finite context is relevant here as well: context is a scarce resource, and every token you add spends some of the model's attention budget Anthropic on context engineering.

One corollary that surprises people: appending a new message to the end of an existing conversation does not invalidate the cache. The prefix (system prompt plus earlier turns) is unchanged, so a multi-turn agent keeps hitting on everything except the newest tokens. Manus reports an average input-to-output token ratio of roughly 100:1 in its agent, which is exactly the shape where prefix reuse dominates cost and time to first response Manus on context engineering.

The delimiter's job is to make the boundary between instructions and data unambiguous to the model and to your own parsers. Three families are in common use.

Markdown fences and rules. Cheap, readable, and fragile. If your data contains triple backticks or a line of dashes, the boundary blurs. Fine for internal tools, risky for anything that accepts public input. LaunchDarkly's prompt engineering list recommends delimiters such as --- or """ to separate parts of a request, which is sound advice as long as you validate that the data cannot contain the same sequence LaunchDarkly's prompt engineering best practices.

XML-style tags. Descriptive tags such as <documents> and <user_input> let the model parse sections by function and give you a stable schema to talk about in the instructions a practitioner's guide to XML tags as delimiters. The catch is escaping: a document that legitimately discusses HTML will contain angle brackets, so either escape the payload or generate a tag name that cannot collide.

Per-request random tokens. The strongest option for untrusted input. Generate a nonce, embed it in the tag name, and state the tag in the instructions above the data.

import secrets

def wrap_untrusted(label: str, payload: str) -> tuple[str, str]:
    nonce = secrets.token_hex(8)
    open_tag = f"<{label}_{nonce}>"
    close_tag = f"</{label}_{nonce}>"
    body = payload.replace(open_tag, "").replace(close_tag, "")
    return open_tag, f"{open_tag}\n{body}\n{close_tag}"

Instruct the model in the stable section to treat everything inside a tag as data, never as instruction, and to ignore any directive that appears inside one. That single sentence is the cheapest injection defense you have, and it costs nothing in cache terms because it lives in the stable part. Structuring a prompt into role, task, instructions and data sections is a widely recommended pattern precisely because it reduces the chance that data is read as instruction LaunchDarkly's prompt engineering best practices.

If you serialize structured data rather than prose, make the serialization deterministic. Manus calls this out explicitly: many languages and libraries do not guarantee stable key ordering when serializing JSON, which silently changes bytes and breaks the cache Manus on context engineering. Sort keys, fix float formatting, and pin your encoder.

Take a support assistant with a 6,000-token stable block (identity, rules, 40 tool schemas) and a 1,500-token dynamic section (tenant config, two retrieved articles, the user turn). Suppose 60,000 requests per day. Round numbers, deliberately simple arithmetic.

If the stable block is truly stable across all requests, the model recomputes the 1,500 dynamic tokens and reuses 6,000 cached tokens on every call. If a timestamp or a per-user greeting sits in line one, the model recomputes 7,500 tokens on every call. The steady-state difference is 6,000 tokens of prefill per request, or 360 million tokens per day that you pay full price for and wait on.

The latency half matters more than the money half for interactive products. Prompt caching cuts time to first token sharply while leaving generation speed unchanged, so a cold prefix means a user stares at a blank panel longer on every single turn Michael Hannecke on prompt caching. One poster on the Hugging Face forums describes exactly this mistake from the other direction: they put long user-specific data at the end of the system prompt assuming they only needed the longest stable prefix within one conversation, then realized the system prompt could be shared across every conversation under their API key a developer's account of cache-aware prompt layout.

The decision rule that falls out: measure your prefix hit rate, not your prompt quality scores. If more than half of your input tokens repeat across requests, caching should be on and your layout should be built for it Michael Hannecke on prompt caching. Then check where the hits stop. If they stop at line one, you have a layout bug, not a model problem.

Session state changes per conversation, so it goes below tenant config and above retrieval. Keep it short and structured: authenticated user id, plan tier, turn number, a handful of flags. Resist the urge to dump a serialized user profile here; trim to the fields the model actually needs, because every token in this block is paid for on every subsequent turn.

Retrieved documents are the hardest block, because retrieval is inherently unstable. Two mitigations are worth the effort. First, fix the number and order of chunks before you format them, so that identical retrieval results produce identical bytes. Second, put stable metadata (document id, title, source URL) before the chunk text inside each document block, so that a chunk whose body changes slightly still shares a prefix with its previous version up to the body. Anthropic's description of context as a finite resource with diminishing returns is the reason to cap retrieval aggressively: more chunks buy less recall than the token count suggests Anthropic on context engineering.

User input goes last, always, wrapped in a nonce tag, and never concatenated into the instruction text. If your architecture puts user text in the system message at all, the only defensible position is that it appears after every rule the model must follow and inside a boundary the user cannot forge.

There is a second argument for this order beyond caching. Errors that enter the context early get referenced and compounded later; a wrong fact in a high-position summary can poison dozens of downstream turns Splunk on context failure modes in production. Keeping volatile content late and clearly labeled gives the model a cleaner boundary between what you asserted and what a retrieval system guessed.

Run these checks in order; each one catches a distinct defect.

1. Diff two consecutive real requests. Serialize both prompts and diff them. The first differing byte should be in a block you consider volatile. If it is in the first 500 bytes, stop and fix the layout.
2. Strip clocks and counters from the stable section. Relative dates ("today is...") and request counters belong with session state, not with the role definition.
3. Verify delimiter collision resistance. Feed your wrapper a payload containing your own closing tag and confirm the output cannot terminate the block early.
4. Sort every JSON payload. Confirm key ordering is stable across runs.
5. Confirm the cache breakpoint lands after the system prompt. Some providers need explicit breakpoints, and the breakpoint should at minimum cover the end of the system message Manus on context engineering.
6. Log the split. Record stable tokens, volatile tokens and cached tokens per request, and graph the ratio.

Step 6 is where most teams discover the problem. The Manus post calls the cache hit rate the single most important metric for a production agent, because it moves both latency and cost at once Manus on context engineering. If you would rather not build that instrumentation yourself, an OpenAI-compatible gateway that reports cache usage per call is the shortest path; Quanta Inference exposes text, image, video, audio and embedding models behind one API key and one prepaid balance (https://quantaintelligence.ai/inference).

The stable section erodes slowly, and each erosion is invisible on its own. Someone adds a locale to the greeting. Someone bumps a tool description. Someone appends a feature flag to the role line. Six months later the "stable" block changes weekly and the cache hit rate is a third of what it was at launch.

Treat the stable block as versioned configuration rather than as a string in a source file. Prompt changes are rollouts: they should be reviewed, tested against a fixed input set, and reversible without a deploy, which is the argument LaunchDarkly makes for managing prompts as runtime configuration LaunchDarkly's prompt engineering best practices. Supercharge's account of building a drill-evaluation system makes a parallel point: prompts should be built from explicit logical blocks, with instructions specifying output format as strictly as possible Supercharge on prompt engineering for reliable systems. Blocks with stable boundaries are easier to evaluate block by block.

Palantir's guidance is a useful reminder of what belongs in that stable text: be clear and specific, specify context, provide examples, set constraints on length and scope, and iterate against outputs Palantir's prompt engineering best practices. Open in the stable layer, tested once, and left alone is the right home for all of it. Anything that must vary per user belongs in the layers below.

When your dynamic section grows complicated enough that layout review needs its own owner, an expert who reads your context and returns a finished prompt architecture plus an evaluation harness is a reasonable next step: Quanta Experts takes the brief and hands back deliverables (https://quantaintelligence.ai/agents).