Setting a Budget for Injection Testing Before Launch

The quick estimate in the short answer is 10 to 20 adversarial cases per tool. Here is how that holds up with numbers.

Take an agent with 12 tools. At 15 adversarial cases per tool, that is 180 adversarial cases. Add the normal cases you need anyway to catch regressions: the Fin AI guide recommends that adversarial inputs make up 15 to 20 percent of a support agent's test suite, and that a focused set of 50 to 100 high-signal cases catches most launch failures Fin's pre-deployment guide. Those two figures are consistent if your functional cases are lean.

Now add the three cheap categories teams forget:

Argument boundaries. What happens when the agent calls a tool with an empty string, a 10,000-character field, a negative refund amount, or an ID belonging to another tenant. These are not prompt injections in the classic sense, but an injection that steers the arguments of a tool call is the actual mechanism of damage. Budget 2 to 3 cases per writing tool.

Chained calls. One injection that reads a document and then triggers a second tool is worth more to an attacker than either alone. The Paragon benchmark work on tool calling separates tool correctness from task completion, and notes that chained tool tasks are where prompt and description changes move results Paragon on tool calling. Budget a set of 20 to 30 multi-step scenarios that cross tool boundaries.

Permission edges. Cases that log in as a low-privilege user and attempt actions reserved for admins, or that use an agent-owned token to read another user's record. These overlap with OWASP's excessive agency risk OWASP GenAI Top 10 and the OWASP project page.

For the 12-tool example the total lands around 300 cases: 180 adversarial per-tool, 60 boundary and permission cases, and 30 chained scenarios, plus a small functional baseline. That is the number you multiply.

Automated scoring gets you triage. It does not get you sign-off, and the budget should say so.

Split cases into two lanes. The automated lane covers the cases where pass or fail is mechanical: did the agent call a tool outside its allowlist, did it include a canary string in an outbound request, did it return another tenant's record. The manual lane covers judgment calls: did the agent's behavior stay within policy, was the refusal appropriate, did a near miss deserve escalation. Expect automated triage for most cases and human scoring for a minority.

The 10 to 15 minute per case estimate is realistic for a security engineer who has the case script, the transcript, and a rubric in front of them. It is optimistic if the transcript is a 40-turn agent reasoning trace. Two things keep the number honest: log the case ID, intent, expected outcome and actual outcome for each case, as the Fin guide recommends, and cap transcripts at the segment under review.

For budgeting, assume 12 minutes average across manual cases. On the 300-case run above, if half the cases need manual scoring that is 150 cases times 12 minutes, or 30 hours. If a security engineer realistically touches every case for context and triage, budget 60 hours. The spread between those two figures is the single largest uncertainty in your estimate, so write it down explicitly rather than picking a middle number.

Human review is also the steepest cost curve in an evaluation practice, priced in payroll rather than tokens, according to one cost breakdown of agent evaluation ContextQA on evaluation cost. That matches what most teams find: the compute bill is small and the reviewer bill is not.

Injection testing fails when one person does all of it, because the same person writes the cases, runs them and signs them off, and tester familiarity becomes blind spots. Divide it.

Security engineers write and run the adversarial cases and own the risk scoring. The 100 to 150 cases per engineer figure in the short answer works out to roughly 20 to 30 hours per engineer on the 300-case run, which leaves room for remediation follow-up. That ratio assumes the engineer already knows the agent; a new engineer needs an extra day of onboarding that the budget should carry.

Domain experts answer the question no security engineer can: does this tool call violate policy or business rules. A refund agent's tool called correctly with an amount one cent over the allowed refund limit is not a vulnerability, it is a business incident, and only the finance or support lead can score it. Budget a half-day per domain expert per release cycle, usually as one two-to-three hour session on the highest-impact findings.

A red team or external tester brings attack patterns the internal team has stopped noticing. The argument for rotating testers is that testers pattern-match to what they have seen before, and many production programs run internal testing monthly with an external specialist quarterly or semi-annually Accelate's guide to ongoing testing. For a pre-launch budget, an external scope of one or two days is enough to pressure-test your cases and rubric.

A quick staffing table for the worked example:

The engineering column is the one teams underbudget. The deterministic output filter that held in testing against more than 20,000 adaptive attacks is application code, not model behavior the Swept AI evaluation. Someone has to build it, and it needs its own tests.

The figure you take to a manager is cost per release cycle, and it has five parts.

Labor. Multiply hours by a loaded hourly rate your finance team gives you, not by salary divided by 2,080. Labor dominates the total.

Judge or scorer compute. If you use a model as a first-pass scorer, every case gets a bill. One published figure puts a single PaperBench run with an LLM judge at roughly $9,500 and a six-model comparison above $150,000, which is why scoping the optional lane matters ContextQA. Your injection run will be far smaller, but running the full corpus against four models multiplies it by four.

Sandbox and API spend. Injection tests must not touch production. Every case needs a disposable environment, seeded fake data, and a canary endpoint you control so you can see whether data left the sandbox. An agent that retries and over-fetches burns tokens, so set a per-case token ceiling and log it; task efficiency is a legitimate part of the test Paragon.

Tooling. If you use a red-team scanner such as Promptfoo's OWASP LLM preset, you pay for the tool or for the engineer time to run it Promptfoo's OWASP LLM guide. Its risk scoring follows CVSS-style severity bands, which gives you a familiar number to report Promptfoo risk scoring.

Maintenance. The corpus needs pruning as the agent changes. Budget a few hours per cycle to retire stale cases and add cases for new tools, since every new tool or integration should trigger an out-of-cycle pass rather than wait for the calendar.

For the 12-tool example, a full cycle is roughly 100 hours of labor plus sandbox and API spend. Three cycles before launch is not three identical runs: run the full 300-case corpus once, then run a regression subset of maybe 80 high-severity cases for the next two. That cuts total hours by about 40 percent without losing the findings.

# Cheap sizing helper: run before you commit a budget number.
tools = {"search_tickets": 12, "read_email": 15, "send_email": 12,
         "fetch_url": 18, "sql_query": 10, "update_crm": 15, "file_upload": 12}
per_tool = sum(tools.values())
boundary = 3 * len(tools)          # argument edges on writing tools
chained = 25                        # cross-tool scenarios
permissions = 20                    # low-privilege, cross-tenant
total = per_tool + boundary + chained + permissions
manual_share, minutes = 0.5, 12
hours = total * manual_share * minutes / 60
print(total, "cases;", round(hours, 1), "manual scoring hours")

If you are running the same corpus across several models to compare behavior, the compute side gets easier to control when every model is behind one billing relationship, which is what Quanta Inference is built for: text, image, video, audio and embedding models through one OpenAI-compatible API key, prepaid in US dollars.

The budget question is really a decision rule, and a decision rule beats a debate. Use three tiers.

Tier 1: the agent can act on the world. It writes, sends, pays or deletes. Full corpus, two security engineers, domain expert sign-off, external red team, and a block on launch until every critical-severity finding is fixed and retested. This is where the lethal trifecta applies and where the risk of real damage lives Airia's practical breakdown.

Tier 2: the agent reads and answers only. No external send, no writes. Run the per-tool corpus, one security engineer, an automated lane for most cases and manual scoring for indirect injection via retrieved content. Chained scenarios only where retrieval feeds a tool call.

Tier 3: internal, trusted users only. A reduced corpus and an explicit note that this is a temporary posture. The Swept AI authors recommend restricting systems to internal trusted personnel until deterministic safeguards are in place and verified, so write that constraint into the launch plan rather than leaving it implied arXiv.

Two rules cut across all tiers. First, score by outcome, because a model can recognize an injected instruction and still act on it under pressure Accelate. Second, treat any defense that lives only in the prompt as untested until you have tried to break it with an adaptive attacker, since static attack sets understate the risk Swept AI. That is where OWASP's ranking of prompt injection as the top LLM application risk comes from OWASP GenAI and why the ranking persists across versions Promptfoo.

For a worked example at Tier 1: 12 tools, 300 cases, two engineers at 60 hours combined, four rounds of fixes and retests at 10 hours each, plus 20 hours of harness work. That is roughly 120 hours before go-live. At three release cycles a year, the steady-state budget is around 40 to 50 hours per cycle once the corpus is mature and the automated lane is doing its share.

If you want the sizing and staffing decisions reviewed before you commit headcount, Quanta Consulting offers a free 30-minute discovery call to walk through your agent's tool inventory, goals and a tailored recommendation, with no obligation.