How to Test a Refund Agent Against Duplicates

Treat refund testing as a matrix of request shapes against system states. The table below is a practical starting set. Each row should become at least one automated test, and the "failure it catches" column is what you assert against, not just the HTTP status.

Two rows deserve special attention. The conflicting duplicate (same key, different payload) is the attack surface of any idempotency scheme: a system that returns the cached response without checking the request body lets a caller reuse a key to mask a different, larger refund. The concurrent duplicate is where most implementations fail in practice, because the check-then-insert pattern has a gap between the check and the insert. The fix is a unique database constraint on the idempotency key, returning a 2xx for the duplicate rather than an error, a pattern documented for webhook processing (Hookdeck).

CREATE TABLE refund_requests (
  idempotency_key text PRIMARY KEY,
  charge_id       text NOT NULL,
  amount_minor    bigint NOT NULL CHECK (amount_minor > 0),
  status          text NOT NULL DEFAULT 'pending'
);

CREATE TABLE refunds (
  id              text PRIMARY KEY,
  charge_id       text NOT NULL,
  amount_minor    bigint NOT NULL,
  provider_id     text UNIQUE
);

With the unique key in place, your duplicate test becomes a real test of the database, not of application logic that might drift.

Partial refunds are where agents get creative and where the ledger gets hurt. A customer returns one item from a three-item order; the agent refunds that line plus tax. Then a second item is returned. The agent must compute the remaining refundable amount from the charge, not from the customer's stated total.

The safest implementation keeps a computed balance: refundable = captured_amount - sum(successful refunds). Every refund request is checked against that balance inside a transaction that locks the charge row. Two tests follow directly. First, a happy-path sequence: on a 100.00 charge, refund 30.00, then 70.00, and assert the refundable balance is zero and both refunds exist. Second, the over-refund: on the same charge, refund 30.00, then attempt 80.00, and assert rejection with an error that names the remaining balance.

That error message is part of the contract. An agent that receives a generic "refund failed" will often retry with a smaller amount, or hallucinate a different charge. Return structured errors with the remaining refundable amount so the agent can correct itself deterministically.

Then test the accounting side. Refunds are not just payment-provider objects; they post to a ledger, and reconciliation systems exist precisely because payments and records drift (HighRadius). Your tests should assert that the sum of refund ledger entries for a charge equals the sum of provider refunds for that charge, and that both are less than or equal to the original charge. Run that assertion after every test in the suite, not just the accounting ones.

The dangerous case is not a failed refund; it is an unknown refund. The provider call times out after the money has already moved. The agent does not know whether the refund exists, and a naive retry creates a second one. PayPal's documentation addresses exactly this: you can retry idempotent calls that fail with network timeouts or 5xx responses, as long as the server still stores the ID (PayPal).

To test it, inject failures at each boundary and assert the end state rather than the intermediate one:

1. Timeout after the provider accepted the request but before your service recorded the response. Retry with the same key; assert one refund.
2. Timeout before the provider accepted anything. Retry; assert the refund eventually succeeds exactly once.
3. Webhook delayed or dropped. A refund succeeded at the provider but your database still says pending. A reconciliation job should query the provider by idempotency key or refund ID and repair the row.
4. Out-of-order events: refund.created arriving before your own request record is committed. The handler should be order tolerant and produce the correct final state.

That fourth case is explicitly part of a proper payment webhook test matrix, alongside signature checks and replays (qaskills.sh). For refunds specifically, also test the case where the agent's own internal step (for example, updating the order status to "returned") fails after the refund succeeded. You need a compensating action that reverses the partial work without reversing the refund, since the money movement is the one thing you cannot blindly undo. The compensating transaction pattern exists for exactly this situation and warns that undo steps must account for concurrent changes rather than restoring a snapshot (Microsoft Azure Architecture Center).

A practical check: after every fault-injection test, run the same ledger assertion from the previous section. If the sums match and the balance is non-negative, the compensating logic did its job.

Example-based tests cover the cases you thought of. Property-based tests cover the arithmetic you did not. The core property is simple to state: for any charge with captured amount C, and any sequence of refund attempts, the sum of successful refunds is between zero and C, and never exceeds C.

Property-based testing frameworks generate random inputs and check that the property holds across all of them. The classic catalog of property patterns includes invariants ("some things never change") and idempotence ("doing an operation twice is the same as doing it once"), both of which map directly onto refunds (F# for Fun and Profit). Here is the shape of a refund property suite:

from hypothesis import given, strategies as st

@given(
    charge=st.integers(min_value=1, max_value=10_000_000),
    attempts=st.lists(
        st.tuples(st.text(min_size=1), st.integers(min_value=1, max_value=10_000_000)),
        max_size=20,
    ),
)
def test_refunds_never_exceed_charge(charge, attempts):
    ledger = RefundLedger(captured=charge)
    for key, amount in attempts:
        ledger.request_refund(idempotency_key=key, amount=amount)
    total = ledger.total_successful_refunds()
    assert 0 <= total <= charge

Three properties are worth encoding explicitly:

• Bound. Total refunded is never negative and never exceeds the captured amount. This catches rounding errors, currency-conversion mistakes and double counting.
• Idempotence. Replaying any accepted request with the same key leaves the total unchanged. Generate a random valid request, apply it twice, assert the total after one equals the total after two.
• Additivity across splits. For a charge of C, any partition of the refundable amount into parts that sum to C produces the same final total as a single full refund. This catches state bugs where sequential partials accumulate differently from one full refund.

Add a fourth property for the agent's own output: for any charge and any natural-language refund instruction, the amount the agent proposes is either zero or at most the remaining refundable balance. That converts "does the agent behave" into a checkable arithmetic property rather than a vibe.

Numbers make the decision rule concrete. Take a charge of 100.00 USD captured in minor units, so 10,000. Run the matrix above and you should see the following end states:

The decision rule for your CI gate is short: a refund suite passes only if every row ends at its expected total, no row exceeds 10,000, and the ledger-to-provider reconciliation matches for every charge touched. If any row over-refunds, the suite fails regardless of how many other tests passed, because over-refunding is the failure that costs real money and is hardest to claw back.

One more practical rule: do not assert on exact error strings from your provider. Assert on your own structured error codes and on the resulting ledger. Providers change messages; your invariants should not.

Refund agents often sit in front of more than one model, because you may want a fast model for intent parsing and a stronger one for edge-case reasoning about returns policy. Testing that arrangement means running the same scenario matrix against each model path and diffing the outcomes. A single gateway simplifies this: Quanta Inference exposes text, image, video, audio and embedding models through one OpenAI-compatible API key with prepaid US-dollar pricing, so your test harness can swap models with a configuration change rather than an integration rewrite.

If your team is designing the refund workflow itself, including where human approval gates belong before an irreversible money movement, Quanta Consulting offers a free 30 minute discovery call to map your goals to a concrete architecture.