How to Write a System Prompt That Keeps a Dispute Agent in Bounds

Write a small table first, for humans. It is the artifact reviewers sign off on before anyone touches the prompt text.

The right-hand column is the part most teams skip. A prohibition without a fallback invites improvisation, because the model still has to produce something. A prohibition with a defined fallback gives the model a legal move. Both Visa and merchant-side guidance converge on this: representment succeeds when evidence directly addresses the reason code, and the wrong evidence loses automatically (Chargeblast on evidence packets). An agent that invents evidence is worse than one that returns an incomplete packet and asks a human.

Convert the table into prompt rules in a fixed shape. Use "Never" for the prohibition, name the source of truth, and name the fallback.

NEVER invent or infer a value for these fields:
- reason_code: source is the dispute notice fetched by the
  get_dispute() tool. If get_dispute() fails or returns no
  reason_code, respond with status "blocked_missing_field",
  name the field, and stop. Never classify a dispute from the
  customer's narrative.
- delivery_date: source is the carrier tracking record in
  shipment_events. If no delivered event exists, write
  "no delivery confirmation on file". Never estimate a date.
- auth_result: source is the 3ds_result field in the payment
  record. If null, write "no authentication result recorded".
  Never write "authenticated" or "3DS passed" without a value.

Two structural choices matter here. First, prohibitions come before the task description in the prompt, so the agent reads constraints before it reads ambitions. Second, every source is named as a tool call or a record path, not as a concept. "From the dispute record" invites the model to reason about what such a record probably says. "From the reason_code field returned by get_dispute()" does not.

Rules state the boundary. Examples show the model what it feels like to stand on each side. For every never-invent field, write two short reasoning traces: one that stays inside the bound, one that crosses it. Keep the disallowed one realistic, because a strawman example teaches nothing.

Allowed example for a missing delivery date:

Order 4471 shows a shipment created on 3 March and a carrier scan "in transit" on 5 March. No delivered event exists in shipment_events. The customer claims the order never arrived. The correct evidence for a merchandise-not-received claim is delivery confirmation, and there is none to cite. I will report "no delivery confirmation on file", note that the last carrier event is the 5 March in-transit scan, and hand the case to an operator rather than asserting delivery.

Disallowed example for the same case:

Order 4471 was placed on 3 March with standard shipping, which normally arrives in three to five business days. The order was likely delivered around 7 or 8 March, so I will state that delivery occurred on 8 March.

The second trace is the one models produce by default: it uses a plausible rule of thumb (typical transit time) to fill a gap, and the number looks authoritative. Naming the failure mode in the prompt helps more than abstract warnings. Say explicitly that "typical" or "expected" timing is not a delivery record.

Allowed example for authentication:

The payment record returns 3ds_result: "authenticated". I can state that the transaction was authenticated under 3D Secure and cite the payment record. I will not characterize the fraud outcome beyond what the record shows.

Disallowed example:

The card was authenticated, so the chargeback will likely be reversed because 3D Secure shifts liability.

The second trace adds a prediction that the record does not support. Liability shift has conditions attached, and the agent cannot know whether any apply. If the case later goes to arbitration, a sentence the agent invented becomes something a reviewer must retract. A useful rule of thumb for writing these pairs is to keep each trace under 120 words and to make the disallowed version do something specific: estimate a date, pick a reason code, add an outcome prediction, or paraphrase a policy. Those four moves cover most leakage.

The broader prompting pattern behind this is documented in practice: ask the model to first identify which facts it needs, then state whether the inputs contain them, then answer or decline (OpenAI community on hallucination-resistant prompts). For a dispute agent, that sequence becomes the response contract, with an explicit decline branch. The thread notes that even grounded prompts occasionally add speculation, which is why the sequence reduces hallucination rather than eliminating it. Plan for a checker regardless.

A single instruction block that says "gather facts, then write the packet" gives the model room to slide a guess into the draft. Split the work into two labeled phases, and make the field ledger the handoff between them.

PHASE 1 - FACT RESOLUTION
For each required field, output one line:
  field | value | source | confidence
Rules:
- source must be a tool name plus a field path, or the word NONE.
- If source is NONE, value must be "MISSING" and you must add the
  field to missing_fields.
- You may not write free text in Phase 1.

PHASE 2 - WRITING
- Every numeric, date, code or boolean you write in the packet must
  appear in the Phase 1 ledger with a source other than NONE.
- If missing_fields is not empty, write the packet with those fields
  marked [AWAITING OPERATOR] and set status to "incomplete".
- Never resolve a missing field by inference, typical values, or
  customer statements.

This ledger makes the failure visible to code as well as to a reviewer. A simple program can compare every value in the Phase 2 draft against the Phase 1 ledger and reject any token that was not sourced, which converts a fuzzy prompt problem into a deterministic check. The same idea underpins structured output validation: teams score each field of a structured output separately so that reviewers can be pointed at the fields that need attention rather than reading everything (Cleanlab on structured output error detection). You do not need a scoring model for a dispute agent. You need the ledger.

The phase split also helps when the agent runs inside a larger pipeline. If several agents touch a case (one triages, one gathers evidence, one drafts), disagreements multiply and the intermediate reasoning disappears from the logs unless it is serialized (Tianpan on agent disagreement). The ledger is a cheap shared state that makes a disputed value traceable to the agent and tool call that produced it. Multi-agent dispute architectures split triage, evidence retrieval, reasoning and documentation for the same reason: each step needs its own audit trail (TechAhead on multi-agent dispute resolution).

One more structural rule: put the never-invent list in the system prompt, not the user turn. Put the case facts in the user turn, wrapped in a delimiter, and add a line in the system prompt stating that facts appearing outside the delimited case block are not evidence. This blocks a common failure where a customer message inside the ticket contains a sentence like "the charge was made on the 14th" and the model promotes it to an authoritative date. Customer statements are claims, not records, and the prompt should say so in those words.

The test you need is not "does the agent handle a full case". It is "does the agent refuse when a required field is absent". Construct a fixture set from a handful of real anonymized cases, then generate variants where exactly one required field is removed or nulled. Each variant carries the expected result: either the safe fallback string or a blocked status naming the missing field.

Run each variant several times at the temperature you plan to ship. Non-determinism is real even at low temperature, and a single pass proves little. Track a refusal rate per field, and treat any variant that leaks a value in even one run as a blocker. Add a second class of test that is subtler: the missing field is absent from the tool response but named in the customer's message. This catches the case where the model treats a user-provided guess as the source of truth.

Keep a regression log of every leak you find in production, and convert each one into a new fixture and, where useful, a new disallowed example in the prompt. The examples and the tests should grow together. A prompt with twenty tested refusal cases and ten matching disallowed examples is a different artifact from a prompt with a bullet list.

Finally, instrument the output. Every packet should carry the Phase 1 ledger, the list of missing fields, and a status. Operators reviewing a batch can sort by status and see which cases are genuinely blocked instead of scanning prose. When a dispute later escalates, the ledger is the record of what the agent knew and what it refused to supply, which is exactly what an auditor or a network reviewer will ask for when a case is contested (Chargeback Gurus on representment).

Take a card-not-present dispute where the customer says the card was used without authorization. The merchant's gateway returns AVS match, CVV match, and a 3D Secure result of "authenticated". The tracking record shows a delivered scan with a signature.

A bounded agent produces a ledger in which reason code, amount, authentication result and delivery record each carry a source and a value, and produces a packet that cites them. It makes no claim about the probable outcome. A merchant side reviewer can then decide whether to contest.

The unbounded version of the same agent produces something that looks better and is worth less. It writes a narrative that says the card was authenticated, implies liability shift, adds a delivery date estimated from shipping speed, and classifies the claim as Visa 10.4 without the dispute notice. If the actual code turns out to be an authorization issue rather than fraud, the whole packet argues the wrong point, and the merchant has spent the response window on evidence that does not address the claim. Card network rules tie the deadline and the required evidence to the code, so a misclassified dispute is not a small error (Rivero on reason codes). With Visa publish

The go