Separating Prompt Changes from Code Deploys

The first structural move is to get prompt text out of the application build. Hardcoding prompts in application code violates separation of concerns and means every wording change needs a redeploy; a registry lets you hot-fix and roll back without shipping a binary, and lets product and domain experts view versions in a UI (the same guide).

The second move is immutability. A version, once created, is never edited. Any change creates a new version. That is what makes logs debuggable: when you trace a bad answer to checkout-agent version v17, you know v17 still means what it meant when the trace was recorded.

What belongs in the artifact is more than the template string. A version that captures only the text is not reproducible, because the same text against a different temperature or model behaves differently. A versioned prompt asset should snapshot the template with its input variables, the model identifier, the sampling parameters, the tool or function definitions available to the model, and metadata such as the author and the originating commit (a prompt-versioning guide).

Application code then asks the registry for a version ID: something like get_prompt("checkout-agent", version="v17") or, for rollout, get_prompt("checkout-agent", alias="production"). Keep one code path that resolves the ID so the source of truth is never ambiguous.

The most common registry mistake is treating an alias like production as a document you edit. If production is a moving pointer that always means "the newest thing," then every save is a release to all users. Aliases are fine as names, but the thing behind the alias should be an immutable version that a human deliberately pointed at.

The practical rule: code refers to versions by ID in tests and pinned paths; runtime resolves an alias to a version ID; the alias changes only through a promotion action that is itself logged. The alias table is small state, and its every change is a release event you can list.

# prompt registry entry (illustrative shape, not tied to a vendor)
prompt: checkout-agent
version: v17
template_file: checkout_agent/v17/template.txt
variables: [cart_summary, shipping_region, policy_excerpt]
model: <model identifier recorded at creation time>
params:
  temperature: 0.2
  max_tokens: 900
provenance:
  commit: 4c1a9e2
  author: support-lead
  ticket: SUP-4411
evaluation:
  golden_set: gs-checkout-3
  run_id: eval-2024-...
  result: pass
aliases:
  production: v17
  staging: v18

Two habits make this safe. First, promotion to production requires a passing evaluation run recorded in the artifact. Second, the reverse move down to staging or canary is a promotion too: a bad version is reverted by pointing the alias back at the previous ID, not by editing text until it works. That gives you a rollback story with a recovery time measured in seconds, which is the same argument used for modern rollback strategies where reversion is a pointer change rather than a redeploy (Octopus).

This is where the neighboring decision becomes concrete. Teams that adopt trunk-based development rely on feature flags to keep the trunk deployable while unfinished work sits behind a toggle; JetBrains notes that managing those flags is a real cost, and that flags are the standard way to hide work you do not want visible yet (JetBrains). Fowler's catalogue draws the same distinction in more detail, sorting toggles into categories because each has a different lifespan and a different owner (Martin Fowler).

Prompt changes should not ride on a code deploy flag. A code flag is owned by the engineering team, scoped to a release, and often removed once the feature is fully on. A prompt flag is owned by whoever owns answer quality, has no code to remove, and may keep running for the life of the feature. Combining them means a prompt revert requires touching an engineering deployment concern, and a code revert silently moves prompt traffic.

The separation pays off in incident response. If quality drops and no code flag changed, the prompt flag history is a short, readable list of version changes with timestamps, and the last entry is the first suspect. If a code deploy coincides with a prompt promotion, your incident timeline can at least show both as distinct events rather than one.

Rollout mechanics carry over from ordinary flags. Fowler's article is worth reading for the pattern of decoupling decision points from decision logic, so the call site asks "which version should I use" and a separate component answers. For prompt rollouts you can use the same progressive delivery shapes as feature releases: an internal cohort, a small slice of traffic, then full exposure, with a kill switch at each stage (Harness). Harness's guidance on flag hygiene applies directly too: human-readable names and clear ownership, or you get a wall of unlabeled flags and stale cohorts when the registry grows past a few dozen prompts.

A prompt change that improves one edge case can break three others, and without a baseline snapshot teams cannot tell whether they improved anything (the prompt-versioning guide). The gate is a golden set: a fixed collection of inputs with recorded expected outputs, run against every candidate version before promotion.

Four rules make the set useful instead of decorative:

1. Freeze the set. Same inputs for every candidate. If the set grows, keep a versioned baseline and re-run the previous production version against the new set before switching, so you are not comparing scores from different datasets.
2. Tie expected outputs to a model and parameter set. A recorded expected output is only meaningful against the model, temperature, and tools that produced it. Change the model and the expected outputs are stale; this is why the model identifier belongs in the artifact.
3. Score against the current production version, not an absolute pass mark. Absolute thresholds drift as the set gets harder. A simple rule: a candidate ships if it is not worse than production on every check group and better on at least one that matters.
4. Record the run ID on the version. The promotion action reads it.

Worked example, with made-up numbers to show the arithmetic. A support triage prompt runs about 40,000 requests a day at a fully loaded cost of $0.0015 per request, so roughly $60 per day, about $1,800 per month. The golden set has 400 cases. Each evaluation call with a long input averages $0.002, so one full run costs about $0.80. Six candidate iterations in a month cost under five dollars, which is negligible against the monthly spend and trivially cheaper than a day of degraded quality.

The stability point matters as much as the average. Sampling makes single runs noisy, so run each candidate two or three times and look at variance, not just the mean. A candidate that scores slightly better on average but flips answers on borderline cases is a regression risk in production even if the headline number improved.

Expect to run the evaluation in your CI system rather than a dashboard. GitHub Actions workflows can run on push, a schedule, or an external dispatch event, and report per-check results back on the change (GitHub Docs). That makes a prompt candidate a first-class pull request: the version gets built, the golden set runs, and the promotion step is blocked unless the run passes.

The change log is what turns all of the above into something an incident reviewer can use in five minutes. For every promoted version, record: the commit that produced it, the author and the reason, the evaluation run ID and the checks it passed, the alias movements and timestamps, the cohort sizes, and any related deploys that happened in the same window.

This log also answers a question that comes up in every review: "was the prompt the cause, or did the app logic change?" Tying prompt versions to application versions makes that answerable, which is exactly the argument for tracking prompts alongside app versions rather than in a separate world (Microsoft Q&A).

A useful habit from artifact registries applies here too: version immutability plus a clear provenance chain from build to deploy is what makes traceability possible at scale, and the same discipline transfers to prompt artifacts (Harness). If your code deploys already carry an image digest linked to a commit, add the prompt version ID beside it so one query returns both.

Two operational details are easy to skip and hard to recover from later. First, tag flags and prompts with an owner so the on-call rotation knows who to page when a cohort misbehaves. Second, retire abandoned cohorts on a schedule: LaunchDarkly's guidance on flag technical debt is blunt that unarchived flags accumulate into a wall of stale state, and an old fallback value quietly serving the wrong prompt version is worse than no flag at all (LaunchDarkly).

If a rollout goes wrong after full exposure, the answer is usually a pointer move back to the last good version, then a re-run of the golden set against the new set additions so the next candidate is better prepared. Rehearse that revert in staging the same way teams are urged to test rollback scripts rather than discovering they are broken during an incident (Octopus).

For teams that want one place to compare versions and run goldens across providers, Quanta Inference exposes text, image, video, audio and embedding models through one OpenAI-compatible API with prepaid US dollar pricing, which keeps model swaps from dragging a second set of credentials and billing into the evaluation loop.