How to Block Prompt Injection on Agent Payment Requests

The first design decision is to stop trying to make the model resistant to every phrasing of an injection. Divide the workflow into five distinct responsibilities, which Payouts' operating guidance for AP agents lays out as interpretation, validation, authorization, execution and reconciliation. The agent may coordinate all five, but it should not own every boundary between them:

The model's job ends at producing a proposed payment in a fixed schema. Everything downstream of that proposal is ordinary code that the model cannot write to. This is the practical form of OWASP's recommendations to define and validate expected output formats with deterministic code and to enforce least privilege by handling privileged functions in code rather than handing them to the model.

The payee must be a closed, structured object. In the invoice extraction schema, allow the model to fill only fields it can ground in the document, and make the routing fields unreachable from the model's perspective. A workable schema looks like this:

{
  "vendor_name_on_invoice": "string",
  "invoice_number": "string",
  "currency": "ISO 4217",
  "amount": "decimal",
  "line_items": [{ "description": "string", "amount": "decimal" }],
  "payee_instruction_from_text": "REJECT_ALWAYS",
  "proposed_payee_id": "vendor_master_id"
}

Two details matter. First, proposed_payee_id is a reference to a vendor master record, not a bank account number. The model never emits an IBAN or account number at all. Second, any bank details found in free text, whether in a line item, a memo, an email signature or an attachment, are captured only as evidence for the fraud log. The schema has no field that reaches the payment service.

The schema plus a JSON schema validator is the first gate. If the output does not parse, or contains a field outside the allowlist, the request fails closed. The agent gets a retry with the original document, not with an expanded permission set.

The vendor master is the system of record for who gets paid. When a supplier is onboarded, a clerk verifies the legal entity and the bank account against independent evidence, and the verified beneficiary record gets a version number. The payment validator then compares the proposed payee and the account on file:

• The vendor name on the invoice must resolve to exactly one active vendor master entry. Ambiguous or fuzzy matches go to a human.
• The beneficiary account used for the payment is always the account stored in the master record, never an account parsed from the document.
• If the invoice text contains any account number that differs from the master record, the invoice is flagged as a suspected fraud attempt and held.

This is the same principle behind Verification of Payee schemes in Europe, where a payer's bank checks that the name and account details match the payee's records before a SEPA credit transfer is processed, returning match, close match or no match, as TIS explains. Payment agents built by providers such as iPiD apply the same idea, converting unstructured payment intent from invoices and messages into data that is then checked against account ownership before execution. Your internal vendor master is a cheaper, stronger version of the same check because it is your own verified record.

A useful decision rule for the validator: any mismatch between the document and the master record on payee identity, account number, currency or payer entity is a hard stop, regardless of how plausible the accompanying explanation is. Explanations are exactly what an injection supplies.

Payee verification handles the redirect attack, but a payment request can be wrong in other ways that an injected line item might also target. Run the deterministic validator over the entire proposed payment:

1. Vendor status. The vendor is active, not blocked, not in a duplicate-payment hold.
2. Amount tolerance. The invoice total is within the agreed percentage of the purchase order or contract value, or an approved change order exists.
3. Currency and entity. The currency matches the vendor's billing currency, and the paying legal entity is the one that contracted with the vendor.
4. Duplicate check. No paid or in-flight payment exists for the same invoice number and vendor.
5. Free-text audit. Extract any account-like strings, URLs or imperative phrases from line items, memos and attachments, log them with the document reference, and confirm none of them influenced a routing field. In our example, GB00 1234 5678 9012 34 lands in the audit log tagged to invoice NW-8841.

Any failure produces a structured exception with a reason code, and the invoice routes to a human queue. The validator is a pure function: same inputs, same outputs, testable without a model in the loop. Write unit tests for it using real attack strings, including the "ignore previous checks" and "already approved by finance" phrasings.

Spending limits are a second layer for the case where something slips through matching. Pre-transaction caps, approval thresholds and rate limits, as described in Locus's agent spending controls, bound the loss from any single runaway or misdirected instruction. FluxA's guide to agent spending limits makes the same point from the failure side: a budget is a policy layer, and without one a manipulated agent can exhaust a balance before a human notices.

Approval must be specific. "Invoice approved" is not an instruction to pay. It is a statement about a liability. Bind the approval to a payment record containing the paying entity, the beneficiary record version, the invoice reference, amount, currency, method, execution window, approver identity, policy version and an expiry. Payouts' control guidance is explicit that a material change should invalidate existing authorization: if bank details change after approval, the earlier approval must not carry forward.

That single rule is the backstop for our incident. Even if every earlier check missed the injected note, the moment the payee differs from the beneficiary version the approver saw, the payment returns for re-approval rather than executing. The approver then sees two different account references on one screen, which is a far easier signal for a human than a wall of invoice text.

Tier the gates by consequence: routine matches inside tolerance proceed under standing authority; anything above a threshold, any new vendor, any beneficiary change and any mismatch pauses for a named approver. Kissflow's guide to approval agents adds two practical details: unanswered approvals should expire and count as a rejection, and the approval state should be persisted so a restart or crash cannot skip the gate.

The strongest control is also the simplest: the model never sees the account number it could redirect to. Give the model the vendor name, invoice reference, partial identifiers for matching and the amounts. Keep full bank details, credentials and one-time codes outside prompts, outside logs and outside memory. RedactSure's fraud guidance frames this around tokenization: a stand-in replaces a sensitive value, and the original is retrieved only through an authorized process at the last responsible moment.

If your agent must confirm a bank change, do it out of band: call the vendor's known contact, not the reply-to address in the email, and record who confirmed and when. A trained clerk comparing two account numbers on a screen performs better than any model reading a paragraph.

Return to Northwind. With the controls in place, the flow runs like this:

The invoice is quarantined, the supplier gets a callback from the finance team using the contact on file, and the incident goes into the threat-intelligence log. The agent did nothing wrong: it read a document, extracted fields, and produced a proposal. The validator, which never trusts text for routing, caught the attempt. That separation is what makes failure recoverable.

Treat the model as an untrusted user when you test. OWASP recommends adversarial testing and attack simulations, including penetration testing that treats the model as if it were an outside attacker, and that discipline applies directly here. Build a regression suite of injection variants: instructions in line items, in PDF metadata, in attachment filenames, in multilingual text, and in "urgent" follow-up emails that reference a fake approver. Run them against the extraction and confirm every one produces either a rejection or a held exception, never an executed payment.

Track a few operational numbers so the controls stay honest:

• Percentage of invoices that clear validation without human review.
• Exceptions per hundred invoices, split by reason code.
• False positives on payee mismatch (legitimate invoices held).
• Time from exception to resolution.
• Count of free-text fields that contained account-like strings.

If payee-mismatch exceptions are near zero, either your suppliers never change banks (unlikely) or your extractor is silently normalising accounts to match the master record. The mismatch rate should be small but nonzero, and every hit should look like evidence of a real problem or a real attack.

For teams building these agents on top of many model providers, running extraction through a single gateway such as Quanta Inference keeps the schema, prompts and validator in one place while the model behind extraction changes. The validator itself should never change with the model.