A Repair Prompt Template That Names the Failing Field

A repair prompt that works has three parts, and each part has a job that the other two cannot do.

The first part quotes the exact fragment the model returned. Not a summary of it, not a description of the error, but the actual bytes. Models are good at patching a concrete, named mistake and bad at guessing why an opaque retry keeps failing the retry write-up. If you paraphrase the output, you have destroyed the evidence the model needs.

The second part names the failing path. A path like items[2].price tells the model where to look, and it tells your own harness where to check afterwards. Without a path, the model may rewrite the whole object, and you lose the guarantee that everything else stayed put. Libraries that expose failed attempts for debugging treat the raw completion as the important artifact precisely because the location and the value have to be preserved together Instructor's retry documentation.

The third part states the expected shape: a type, any constraint the validator enforces, and one short example of a valid value. This is where you translate the validator's language into the model's. "Field X must be an integer, you sent a string" is the kind of plain instruction that gets acted on the retry write-up. Ambiguous types are a recurring failure: a phone number returned as an integer drops its leading zero, and a date arrives as a string in several formats across calls MLflow's structured output guide. Stating the type and giving one example closes that gap.

The template below is deliberately boring. Labels in caps, one idea per block, and a closing rule that restricts the model's edit surface. Keep the labels stable so you can assert on them in tests.

You are correcting one field in a JSON object you produced earlier.

RECEIVED
The object below failed validation. Treat it as the current state.
<received>
{"order_id":"A-1043","customer":"Dana Ruiz","items":[
  {"sku":"SKU-11","qty":2,"price":19.5},
  {"sku":"SKU-42","qty":1,"price":7},
  {"sku":"SKU-7","qty":3,"price":"4.25 USD"}
],"total":56.75}
</received>

FAILING PATH
items[2].price

EXPECTED SHAPE
Type: number (JSON number, no currency symbol, no string).
Constraint: greater than 0, at most two decimal places.
Example of a valid value for this path: 4.25

RULE
Change only the value at items[2].price. Return the complete object with every
other key and value byte-identical to RECEIVED. Return JSON only.

Three details carry most of the weight. First, the received block is presented as the current state, which frames the task as editing. Second, the expected shape gives a bare number as the example, not a quoted string, so the type is demonstrated rather than described. Third, the rule says byte-identical, which gives you a testable claim rather than a vague instruction to preserve the rest.

If you generate your prompts from a model definition instead of writing them by hand, derive the expected shape section from the schema rather than retyping it. Pydantic emits JSON Schema from a model with model_json_schema, including types and constraints inside fields, so the prompt and the validator can share one source of truth Pydantic's JSON Schema docs. Where you drive generation through structured outputs with that schema, the model is already constrained to your shape and the repair prompt becomes a second line of defense for value-level errors that the schema cannot express OpenAI's structured outputs guide.

Here is the same case end to end, so you can see what a good repair looks like and what to do when it nearly works.

The received object has a nested array. The third item's price is a string with a currency suffix. The validator rejects it because the schema says number. The repair prompt above goes out with the model's own text in the received block.

A correct response returns the whole object with one difference:

{"order_id":"A-1043","customer":"Dana Ruiz","items":[
  {"sku":"SKU-11","qty":2,"price":19.5},
  {"sku":"SKU-42","qty":1,"price":7},
  {"sku":"SKU-7","qty":3,"price":4.25}
],"total":56.75}

Note what the model did not do. It did not recompute the total even though the total now looks inconsistent with the items, because the rule said only the named path may change. That is the point. Arithmetic corrections belong in your code, not in a repair prompt, and mixing the two is how you get silent data corruption. If your pipeline needs the total recomputed, do it after validation passes, deterministically.

The near miss is instructive. A repair that returns price: 4.25 but also reformats order_id from A-1043 to a nested object, or drops a trailing key, passes a type check and fails your preservation check. This is why the test harness in the next section compares the whole object rather than just the repaired field.

There is a boundary case worth planning for. If the failing path does not exist in the received output, the model cannot edit it, and the repair prompt will either invent the path or return the object unchanged. Detect that case in code before you send the prompt and fall back to a full regeneration with the schema, not a repair. The same applies to output that is so truncated it cannot be serialized back into the prompt; a repair needs parseable material to work on, and truly empty or truncated output has nothing to correct the retry write-up.

For syntactically broken JSON (single quotes, missing braces, trailing commas), a repair prompt is the wrong tool entirely. Use a local repair library first: jsonrepair fixes missing closing brackets, single quotes, unquoted keys and trailing commas without a model call the jsonrepair cookbook, and Python has an equivalent in json_repair with the same remit the json_repair repo. Repair the syntax locally, then use the prompt for semantic errors. The cookbook is explicit that repair is best-effort and that the object must still match your schema afterwards the jsonrepair cookbook.

The harness below verifies the template itself, not the model. It renders the prompt for three known-bad responses, asserts that each required part appears, calls your repair function, and checks two conditions: the output parses and validates, and every field outside the named path is byte-identical.

import copy, json
import pytest

def render_repair(received_obj, path, expected):
    return TEMPLATE.format(
        received=json.dumps(received_obj, separators=(",", ":")),
        path=path,
        expected=expected,
    )

def get_path(obj, path):
    cur = obj
    for tok in path.replace("]", "").replace("[", ".").split("."):
        if tok == "":
            continue
        cur = cur[int(tok)] if isinstance(cur, list) else cur[tok]
    return cur

def set_path(obj, path, value):
    toks = path.replace("]", "").replace("[", ".").split(".")
    cur = obj
    for tok in toks[:-1]:
        cur = cur[int(tok)] if isinstance(cur, list) else cur[tok]
    last = toks[-1]
    if isinstance(cur, list):
        cur[int(last)] = value
    else:
        cur[last] = value

def assert_only_path_changed(before, after, path):
    expected = copy.deepcopy(before)
    set_path(expected, path, get_path(after, path))
    assert json.dumps(expected, sort_keys=True) == json.dumps(after, sort_keys=True)

BAD_CASES = [
    ("items[2].price", {"items": [{"price": 1}, {"price": 2}, {"price": "4.25 USD"}]}),
    ("items[2].price", {"items": [{"price": 1}, {"price": 2}, {"price": "4,25"}]}),
    ("items[2].price", {"items": [{"price": 1}, {"price": 2}, {"price": None}]}),
]

@pytest.mark.parametrize("path,bad", BAD_CASES)
def test_template_and_repair(path, bad):
    prompt = render_repair(bad, path, "number, > 0, example 4.25")
    for label in ("RECEIVED", "FAILING PATH", "EXPECTED SHAPE", "RULE"):
        assert label in prompt
    assert path in prompt
    assert json.dumps(bad, separators=(",", ":")) in prompt
    out = repair(prompt)                 # your model call, temperature 0
    parsed = json.loads(out)
    assert isinstance(get_path(parsed, path), (int, float))
    assert_only_path_changed(bad, parsed, path)

The third assertion is the one people skip. Checking that the repaired field is a number is necessary and not sufficient; the preservation check is what catches a model that tidied the rest of the object while it was there. Run the suite on every prompt change, and keep the three bad cases in your repository as fixtures, because they encode the exact failures you already saw in production.

To widen coverage without writing cases by hand, render a batch of deliberately corrupted outputs from your fixtures and grade the pass rate. Deterministic assertions such as JSON validity, schema validation and exact equality are exactly the checks a prompt evaluation suite should run before a template reaches traffic Promptfoo's deterministic metrics.

Not every failure deserves a repair prompt. This rule keeps you from spending a model call where a function call would do.

The rule in one sentence: repair when the failure is one locatable value in parseable output, regenerate when the shape is wrong, and fix locally when the syntax is wrong. Cap repair attempts at one, occasionally two. Do not count a provider swap as a real attempt, because prompt semantics do not always transfer across providers mid-loop the retry write-up.

A word on field descriptions, because they are the cheapest permanent fix. In the OpenAI forum thread on Pydantic and JSON schema, the original poster's translation output occasionally came back with the wrong key shape or wrapped values, and the thread's practical advice is to be more explicit with fields and to give them natural language descriptions that travel into the schema the OpenAI forum thread. A description that says "ISO 8601 date string, for example 2026-03-15" prevents a whole class of repairs before they are needed.

Two layers sit around the repair prompt. Upstream, prefer schema-constrained generation over prompted JSON, because it removes a whole failure class at decode time rather than reducing its rate. Structured outputs guarantee adherence to the supplied schema, so the model cannot omit a required key or invent an enum value OpenAI's structured outputs guide, while JSON mode only guarantees syntactically valid JSON that may still miss keys or change types MLflow's structured output guide. Downstream, log both raw and parsed outputs, track a validation failure rate, and alert when it moves; silent regressions in structured output are common and expensive to trace after the fact MLflow's structured output guide.

There is a schema-side decision that reduces repair volume more than any prompt tweak. Keep the schema minimal: required fields, strict types, enums where the set is closed. Rich Pydantic schemas with optional fields, numeric bounds and unions do not always survive translation into a provider's strict mode, and the sanitizing people write to bridge the gap is where subtle mismatches creep in the LangChain schema sanitizer thread. A schema that validates cleanly in your own code and maps cleanly to the provider produces fewer value-level errors, so fewer repairs.

When the failing output is a tool call rather than a document, the same three-part structure applies to the arguments. Function calling returns JSON arguments that must match the tool's parameter schema, and the schema is what the model is working from OpenAI's function calling guide. Naming the failing argument path (arguments.location) is as effective as naming a document field the function calling primer.

If your repair path has to cover generators and graders across several vendors, keep the prompt in one repository and standardize the client. An OpenAI-compatible endpoint that serves every kind of model behind one key, with prepaid billing in US dollars, means the harness above tests one prompt rather than one per provider: https://quantaintelligence.ai/inference.