Forecasting the Support Load of a Default-On AI Feature

Do this in your helpdesk export, not in a slide.

1. Define the cohort precisely: users who had the feature enabled for at least seven continuous days during the window.
2. Count active users in the cohort per week. Use your product analytics session or event table, not your billing table, because billing counts accounts that never open the product.
3. Count the feature-attributed tickets created in the same weeks.
4. Divide tickets by active users, multiply by 100, and average the four weekly values.

A worked example, with round numbers for clarity:

The mean is 5.3 contacts per 100 users per week, but the trend is downward across all four weeks. The decision rule here matters more than the number: if the week-four value is the lowest and still falling, use week four (4.0) as the base case and week one (7.0) as the stress case. If the trend is flat, use the mean. This gives you a range rather than a false point estimate, and the range is what you bring to the staffing conversation.

Do not use a cohort smaller than 500 users. At 200 users, four tickets versus five in a week swings your rate by 0.5 points per 100, which is enough to flip a staffing decision. At 500 or more, a single anomalous ticket moves the rate by 0.2 points or less, and the noise stops dominating the signal.

The arithmetic is deliberately trivial: default-on tickets per week equals contacts per 100 users per week, times the default-on active user population, divided by 100.

Take a product with 40,000 weekly active users after default-on. At a base case of 4.0 contacts per 100, you project 1,600 tickets per week. At a stress case of 7.0, you project 2,800. That spread of 1,200 tickets is the entire reason to build a range.

The default-on population is not your whole user base. Two adjustments reduce it:

• Geography or account tier exclusions. If the rollout excludes regulated regions or enterprise tenants on a separate release track, subtract them. This is also where a feature flag with percentage rollout earns its keep: you can widen in steps of 5, 10, then 50 percent and watch the contact rate at each step rather than trusting the projection blindly the OneUptime percentage rollout guide. The consistent hashing described there keeps each user in the same bucket across requests, which keeps your experiment clean.
• Engagement decay. A feature that is default-on but ignored generates no tickets. If your cohort used the feature at 30 percent weekly and you expect the default-on population to use it at 15 percent, halve the projected tickets. This is the single most common source of over-forecasting, and it cuts in your favor for once.

A canary stage between opt-in and default-on is the cheapest way to firm up both numbers. Google Cloud's canary documentation describes splitting traffic between an existing and a new version and rolling out to a subset of users before going wide the Google Cloud canary deployment guide. The contact rate measured in that canary is a far better input than any cohort extrapolation.

A total ticket count cannot be staffed. Topic mix can. Pull the feature-attributed tickets from your cohort, read a random sample of at least 100, and assign each to a small taxonomy. Five to seven topics is enough; more than that and your shares become unstable.

A realistic mix for a generative AI feature looks like this:

Two insights hide in this table. First, the largest bucket is often a knowledge problem, not a defect: "does not know how to trigger the feature" at 25 percent is a documentation and onboarding fix, not a hiring need. Cutting that share in half with a better first-run experience removes 200 tickets per week without a single new agent. This is the point the contact rate literature makes about decomposing by intent: an aggregate rate hides which slice is solvable with proactive communication rather than headcount the Decagon contact rate glossary.

Second, hallucination tickets are usually the cheapest per ticket to triage and the most expensive to resolve, because they require reproduction and a knowledge base update. Topic mix only becomes useful when it is combined with resolution time, which is the next step.

Multiply each topic's weekly ticket count by its average resolution time in hours. Use your helpdesk's actual handle time plus after-call work, not a benchmark. If you have no measured figure, run a time study on 30 tickets per topic before you publish a forecast.

Now you have the number your capacity plan needs: 1,708 agent-hours per week at the base case. Multiply by the stress case ratio (7.0 divided by 4.0) and you get roughly 2,990 hours. That range is what you take into the gate.

The general formula, stated once: agent-hours per week equals the sum, over all topics, of topic tickets times topic resolution time. Supportbench makes the same multiplication as the core of its workload estimate, multiplying forecast volume by average handle time, then dividing by agent availability and adjusting for shrinkage such as breaks, training and absence the Supportbench staffing guide. Gorgias notes shrinkage is often far larger than teams assume and puts typical industry figures in ranges that erode available hours well below headcount times forty the Gorgias forecasting framework.

Do the shrinkage arithmetic honestly. One full-time agent is nominally 40 hours per week, but after vacation, sick time, training, meetings and breaks, available hours are commonly 60 to 85 percent of that. At 75 percent availability, one agent offers about 30 productive hours. The base case of 1,708 hours therefore needs roughly 57 agents on the feature alone if this were the only workload they carried. In practice the feature shares agents with everything else, so the useful figure is the incremental hours, which you then compare against slack in the existing team.

The gate is the governance mechanism. Without a pre-agreed threshold, the default-on date becomes political and support finds out on launch week.

Write the gate down before the widening decision and share it with product, support and finance:

• Green. Projected agent-hours at the base case are below 60 percent of available feature capacity. Proceed with default-on at the planned date.
• Amber. Projected hours land between 60 and 80 percent of capacity. Proceed only with a documented mitigation: new knowledge base articles for the top deflectable topic, temporary overtime, or a staged rollout over two weeks.
• Red. Projected hours exceed 80 percent of available capacity at the base case, or exceed 100 percent at the stress case. Delay the default-on rollout or add staff first. No exceptions.

Use the number, not the narrative. If the base case lands at 1,708 hours against 1,900 available, you are at 90 percent and the gate says red, even if the stress case is comfortable in the conversation. The gate exists precisely to override the optimism that gathers before every launch.

Two things strengthen the gate. First, monitor the actual contact rate weekly during the staged rollout and recompute; if the observed rate exceeds the base case by more than 20 percent for two consecutive weeks, roll back the percentage to the previous step. Second, keep a fallback: a feature that can be disabled by config, not by redeploy, is the only way the rollback clause is credible. GrowthBook's guide to feature flagging AI models makes the case that model IDs, prompts and parameters belong behind flags because they are otherwise invisible to your deployment pipeline the GrowthBook feature flagging guide, a point echoed in Microsoft's own guidance to keep a fallback path during a staged rollout rather than treating the staged configuration as permanent the Microsoft Entra staged rollout documentation.

Fill this in once and reuse it every time a feature goes default-on. Copy it into a spreadsheet.

1. COHORT
   users_in_cohort = 500 (min)
   observation_weeks = 4 (min)
   weekly_active_users[w1..w4]
   feature_attributed_tickets[w1..w4]
   contact_rate_wk = tickets / active * 100
   base_case = week4 if declining else mean
   stress_case = week1

2. EXPOSURE
   default_on_wau
   engagement_factor = default_on_usage / cohort_usage
   projected_weekly_base = base_case * default_on_wau / 100 * engagement
   projected_weekly_stress = stress_case * default_on_wau / 100 * engagement

3. TOPICS (sample 100+ tickets)
   topic | share | tickets | avg_res_hrs | hours

4. CAPACITY
   available_agent_hours = agents * 40 * availability
   utilization = projected_hours / available_agent_hours
   gate = green (<60%) | amber (60-80%) | red (>80%)

5. DEFLECTORS
   top_deflectable_topic | share | expected reduction | hours saved

6. DECISION
   date | owner | outcome | review date

Two notes on using it. Run the deflector line before you run the capacity line. If a documentation change plausibly cuts the "does not know how to trigger" bucket by half, subtract those hours from the projection before you decide whether to hire. And review the template after the first staged rollout, because the gap between your projected contact rate and your observed one is the calibration that makes the next forecast credible.

If the manual arithmetic becomes the bottleneck, or you need the feature to call several different models behind one interface without three separate support runbooks, Quanta Inference (https://quantaintelligence.ai/inference) gives you text, image, video, audio and embedding models through a single OpenAI-compatible key, which keeps the number of vendor-specific failure modes you have to staff for down to one.