How to Test a Cancellation Agent Against Proration Errors

A double-entry ledger is a test oracle that writes itself. Every operation must leave total debits equal to total credits, and every transaction must touch at least two accounts with a net of zero. That single property catches the most common proration bug class: a credit issued without a matching charge, or a charge posted twice.

Keep the account chart small and explicit. Five accounts cover both agents:

An upgrade on day 20 of a 30-day cycle looks like this in a ledger you can diff:

[
  {"account": "unused_time_credit", "debit": 3000, "credit": 0},
  {"account": "deferred_revenue",   "debit": 0, "credit": 3000},
  {"account": "cash",               "debit": 5000, "credit": 0},
  {"account": "deferred_revenue",   "debit": 0, "credit": 5000}
]

Thirty dollars of unused value on the old plan, fifty dollars charged for the new plan, twenty dollars net collected. The numbers come from the worked example in Recurly's proration guide, where a customer on a $90 plan upgrades to a $150 plan with 10 of 30 days remaining. A cancellation produces the same shape with the new-plan leg removed, which is exactly why one property suite covers both.

Write the invariant as an executable assertion:

def assert_balanced(entries):
    debits = sum(e["debit"] for e in entries)
    credits = sum(e["credit"] for e in entries)
    assert debits == credits, f"unbalanced: {debits} vs {credits}"
    assert all(e["debit"] == 0 or e["credit"] == 0 for e in entries)

The second line forbids a single entry that both debits and credits the same account, a shape that can hide sign errors. Reconciliation tooling vendors make similar accuracy claims for automated matching, with HighRadius describing 99% reconciliation accuracy as a target for their agent-assisted close, but a test ledger does not need a probability, it needs an exact identity.

Example-based tests encode the cases you imagined. Property-based tests encode the rule and let a generator hunt for the counterexample. The classic taxonomy of property shapes, from "different paths, same destination" through idempotence and the test oracle, is laid out in this property-based testing walkthrough, and each shape maps onto a billing rule. Idempotence, for instance, says that replaying the same cancellation event twice must produce one set of ledger entries, not two.

The generators that matter for proration are the ones that vary the shape of time:

• cycle length in days, drawn from 28 to 366 so February and leap years appear
• cancellation or switch day, drawn from day 1 to the last day of the cycle
• old and new plan prices in minor units, including zero for cancellations
• whether the cycle boundary aligns with a month boundary or not
• the number of seats, for per-seat plans

Two generators earn their keep immediately. Cycle lengths of 28 to 31 days expose rounding differences between a "days in month" and a "fixed 30-day divisor" policy, and switch days at day 1 and the final day expose off-by-one errors where the agent credits a day it also charges for. The same discipline applies to market data validation, where non-negative spreads and monotonic timestamps are treated as definitional invariants rather than statistical tendencies: if the property fails, the pipeline is broken, not the market.

For each generated case, assert three properties on both agents:

1. Balance. Total debits equal total credits after every operation.
2. Conservation. Unused-time credit plus new-plan charge equals the net cash moved, within one minor unit, and any difference lands in rounding_residual.
3. Monotonicity. Repeating the operation with the same idempotency key produces no additional ledger entries.

The beginner's guide to property-based testing at danicfilip.com is blunt about the trade-off: properties take longer to write than examples and run slower, so restrict them to functions where an invariant exists. Proration math is exactly such a function, a pure calculation over days and prices, which makes it cheap to test at volume.

Cancellation and plan changes both arrive as webhook events, and webhooks are delivered at least once. Hookdeck's idempotency guide is explicit that providers operate on an at-least-once guarantee, so your handler will eventually see the same event twice. Your test suite must prove that the second delivery changes nothing.

The test matrix is small but unforgiving. This payment webhook testing guide frames it as four cells: valid versus forged signature, first delivery versus replay, concurrent duplicates, and out-of-order sequences such as a refund arriving before the payment it reverses. Each cell maps to a ledger assertion. A forged signature returns 401 and posts no entries. A replay returns a success status and posts the same entries zero additional times. Concurrent duplicates yield exactly one set of entries, which is why the database needs a unique constraint on the provider event id, and Stripe's idempotent request documentation shows the same idempotency-key pattern at the API layer for your own outbound calls.

Out-of-order events are where the two agents diverge and where the shared harness earns its keep. Send subscription.cancelled before invoice.paid and the cancellation agent should defer the credit until the payment lands. Send the equivalent plan-change pair to the proration agent and it should do the same. If one agent handles the late arrival and the other posts a negative deferred balance, the differential test in the next section flags it even though each agent's own suite is green.

One subtle trap is the idempotency key itself. If the client generates a fresh UUID on every retry, the server sees two distinct operations and the idempotency mechanism protects nothing, as discussed in this Stack Exchange thread on key reuse. Your tests should retry with the same key and confirm the ledger is unchanged.

Differential testing compares two implementations that are supposed to agree. Here the inputs are identical: the same customer, the same billing period, the same plan history, the same effective day. Cancellation is modeled as a switch to a zero-priced plan, so both agents should produce the same unused-time credit for the same day. Compare their ledger entries field by field.

def differential(case):
    cancel_entries = cancellation_agent.run(case)
    switch_entries = proration_agent.run(case.as_switch_to_free())
    cancel_credit = credit_on(cancel_entries, "unused_time_credit")
    switch_credit = credit_on(switch_entries, "unused_time_credit")
    assert cancel_credit == switch_credit, (
        f"mismatch on day {case.day} of {case.cycle_days}: "
        f"cancel={cancel_credit} switch={switch_credit}"
    )

Run it over a thousand generated cases and expect a spread of failures early, especially on the boundary days. Treat every mismatch as a specification question first: decide which agent is right, encode that decision as a new property, then fix the other agent. The compensating transaction pattern is the reference for how to structure the fix, since a cancellation credit is by definition a compensating action that reverses earlier work and must account for concurrent changes rather than blindly restoring prior state, as Microsoft's architecture guidance describes and this design pattern write-up summarizes.

When the two agents truly cannot share code, for example because one calls a vendor billing API and the other computes locally, the differential test degrades to a consistency check: generate a case, run it through both, and assert the net cash movement agrees. A recorded test oracle, the seventh property pattern in the F# guide above, catches regressions as each agent evolves independently.

Every proration incident produces a test case, and the discipline is to convert the specific example into a general property so the whole class of bug is covered. The conversion usually takes one of four forms, which is worth stating as a checklist your team applies during incident review.

The third row deserves emphasis because it is the one accountants notice. Splitting a credit across three revenue categories rarely divides evenly, and every implementation needs a named account for the remainder. A property that says the residual is always posted, and always to that account, is stronger than any number of specific penny fixtures.

Recording the incident as a property also keeps the suite honest over time. Compensating operations and approval gates both change with policy, and the four gate types framework at Digital Applied shows how audit-trail requirements evolve: nine fields captured per gate event, a state diff that auditors can read. Your test suite should assert the gate fields exist too, because a gate that blocks without an audit record is not a control. When an approval gatefront of a refund over a threshold is added, the property becomes "no debit above the threshold posts without a gate record," and the differential test between the two agents runs again.

Put the property suite in CI at two levels. A fast subset with a few hundred generated cases runs on every pull request and must finish in minutes. A slow sweep with tens of thousands of cases, including the longest cycle lengths and the extreme switch days, runs nightly and on release candidates. Property libraries shrink failing cases to the smallest counterexample, so when the nightly run fails you get a specific day and price pair to debug, and that pair becomes a new example test in the fast suite.

Two practical guardrails keep the harness maintainable. Pin the ledger chart to a small, versioned file, so a new account is a deliberate change that forces every property to be re-read. And keep the model calls out of the property loop wherever possible: test the deterministic calculation and the tool contracts with mocked responses, then run a small number of end-to-end cases against a real model to confirm the agent still picks the right tools. The error-taxonomy list in the Trusted Data guide names timeouts, retries, stale state and hidden tool errors as the failure modes that turn a prototype into an incident, and each one has a ledger consequence you can assert.

For teams building these agents on multiple providers, the model-selection question can dominate the test matrix. An expert team that has run the differential harness across several models can tell you which combinations survive the properties before you commit. That is the kind of review a free 30-minute AI discovery call is designed for: your goals, your ledger, and a tailored recommendation.