Setting the Re-ask Limit for a Support Agent Before It Deflects

Start with what "deflected" is supposed to mean. A contact counts as deflected only when the underlying issue is actually resolved, not when the customer abandons before reaching a human (Decagon's definition). Decagon's formula is deflected contacts divided by total contact attempts. In its worked example, 6,500 of 10,000 monthly attempts resolved without a human, a 65% deflection rate.

Over-asking attacks that number from underneath. A customer asked a second and third question they consider redundant may close the window. Under a naive count, that session is deflected. Under a re-contact adjustment it is not. Helply's worked example is blunt about the size of the error: of 450 self-service resolutions in 1,000 attempts, 90 came back within 48 hours, taking the reported 45% down to a true 36% (Helply on deflection). Teams running the adjustment for the first time commonly find true deflection 15 to 25% lower than reported.

Conversational flow research points the same way. Hamming, analyzing voice agent interactions, finds poor flow the top predictor of user abandonment and sets targets of fewer than 6 turns per task, under 3% repeated questions, and under 3 interruptions per call (Hamming's flow framework). A clarifying question is a turn. Two extra questions on a four-turn task puts you at the edge of the target before the customer has confirmed anything.

So over-asking is not a style problem. It is a measurable drain on the metric you are probably reporting upward.

Write the definition down before you instrument anything, because the number is easy to fudge. A workable definition:

The re-ask limit for an intent is the maximum number of clarifying questions the agent may ask in a single conversation before it must answer with a stated assumption or hand off to a human.

Three decisions sit inside that sentence and each changes the measurement.

What counts as a clarifying question. A request for information the agent needs to proceed. A confirmation ("Just to confirm, you want the annual plan?") counts. A restatement of what the customer already said does not, and should be logged as a defect, not as a turn. Hamming's repetition target gives you the threshold: repeated questions should sit under 3% of sessions.

Whether the limit is global or per intent. Per intent, with a global ceiling. A password reset may need one question (which account). A billing dispute may need two (which charge, which month). A single global number either starves the complex intents or indulges the simple ones.

What "reaching the limit" triggers. Not a hard stop on the conversation, but a hard stop on asking. The agent still has two moves: answer or hand off. Build both before you ship the limit, or the agent will invent a third (silence, or a loop).

The decision rule needs two rates per intent, measured across the same window, with the same re-contact adjustment applied to both.

Deflection rate at a limit of n is the share of conversations on that intent, where the agent was allowed up to n questions, that ended without a human handoff and without a re-contact on the same issue inside 24 to 72 hours. Use a fixed window and keep it fixed. Decagon notes the behavioral proxy, a contact is deflected if no human ticket opens within 24 to 72 hours, scales better than surveys but can overstate deflection when a customer gives up and returns on an untagged channel (Decagon).

Over-asking rate is the share of conversations where the last question asked by the agent was answerable from information already in the transcript or the account record. You can label this with a cheap model pass plus a human sample, and it does not need to be perfect to be useful. It needs to be consistent across the three arms.

Run the three arms as a randomized test, not sequentially. If you raise the limit from one to two across the whole base, seasonality and a knowledge base update will contaminate the comparison. Decagon's own product listing includes live A/B testing for exactly this reason (Decagon Experiments).

Worked example. Suppose a "refund status" intent sees 2,000 conversations per arm over two weeks.

The decision rule: choose the largest limit where each extra question still adds deflections without a matching rise in over-asking. Here, moving from one to two questions bought 150 net deflections and 5 points of over-asking. Moving from two to three bought 30 net deflections and 9 points of over-asking, so two wins. The intent's limit is two. A useful default before you have logs is two per intent, but the point of the exercise is that the number is yours.

Note what the netting does. Without subtracting re-contacts, arm three's gross deflection would look close to arm two's, and the over-asking rise would be the only signal you had. With netting, the extra question is visibly buying almost nothing.

You cannot run the three-arm test without step-level telemetry. Agent observability means seeing what the agent did on each request, why, and at what cost, with each user request appearing as a connected trace covering planning, model calls, tool calls and intermediate results (groundcover's guide). For this exercise you need three fields on every trace: the intent, the ordinal of each clarifying question, and the source of the information the question requested (transcript, account system, nowhere).

OpenTelemetry's GenAI work matters here because it standardizes the shape of that telemetry across frameworks, which keeps a prompt-instrumented agent and a graph-based agent queryable the same way (OpenTelemetry on AI agent observability). Microsoft's guidance adds the two pieces traditional observability lacks, evaluations and governance, and recommends continuous monitoring of agent behavior rather than spot checks (Microsoft's observability practices). AWS's prescriptive guidance makes the same case for event-driven agent systems: without observability you cannot correlate a symptom with the step that caused it (AWS Prescriptive Guidance).

A minimal log record for the limit study:

{
  "conversation_id": "c_88123",
  "intent": "refund_status",
  "q_ordinal": 2,
  "question_text": "Which month is the charge from?",
  "info_source": "account_record",
  "already_in_transcript": true,
  "exit_type": "answer_with_assumption",
  "handoff": false,
  "recontact_72h": false
}

The field already_in_transcript is what feeds the over-asking rate. Everything else feeds deflection and the exit mix. If you are running several models behind one agent, route them through a single endpoint so the trace schema does not fork by provider; Quanta Inference exposes text, image, video, audio and embedding models behind one OpenAI-compatible API and one key (Quanta Inference).

The limit is only as good as the two exits behind it. Both need to be built, tested and instrumented before the limit goes live.

Exit one: answer with a stated assumption. The agent picks the most probable reading of the request and says so. "I am treating this as the March charge on the Pro plan; here is how to dispute it. If it is a different charge, reply and I will redo this." This is where the confidence rule and the re-ask limit meet. A calibrated abstention band, derived from your own reliability diagram rather than a guessed threshold, tells you which answers are safe to give with a stated assumption and which are not (USQRD on calibration and the human gate). USQRD's finding is worth repeating internally: self-reported confidence and raw logprob averages did not track correctness in their engagements, while external signals like ensemble agreement and grounding checks did. If your exit-one gate is a self-reported score, expect it to wave the wrong cases through.

Exit two: hand off with context. The handoff should carry the intent, the questions asked, and the assumptions the agent considered. A handoff that makes the customer repeat themselves is worse than a second question. Verify handoff quality directly: the share of handoffs where the human agent re-asks something already in the transcript is a clean, countable defect rate (Thinklytics on deflection metrics).

Some teams route on the model's own confidence at the step level. Research on step-level escalation found that gating each action on an uncertainty estimate recovered much of the gap to a stronger model at a fraction of the escalation rate, though the study was limited to one model family and scripted tasks (step-level handoff research). Treat that as a design option, not a proven default for your traffic.

A limit without a review cadence drifts. The agent gets a new prompt, the knowledge base updates, a policy changes, and the intent mix shifts under a limit that was tuned for last quarter's conversations. Put three numbers on the same weekly view per intent: net deflection, over-asking rate, and repeat-question rate.

The comparison that matters is not against a vendor benchmark but against your own arms. E-commerce deployments with mature AI typically see 55% to 75% deflection, SaaS 40% to 60%, and finance and healthcare 25% to 45%, because contact complexity, not automation skill, sets the ceiling (Decagon). A "refund status" intent at 59% is not lagging a SaaS number; it is a different population. The right question is what the third question buys on that intent, this quarter.

Two guards against gaming:

• Report gross and net deflection side by side. The gap between them is your cost of over-asking, made visible. Pylon notes that AI deflection programs commonly claim 40 to 60% for technology companies, with best-in-class reaching 85% on routine questions (Pylon on AI ticket deflection), and those numbers mean nothing if a chunk of them is abandonment.
• Cap the total turns, not just questions. Hamming's six-turn target does this job. An agent that asks two questions and then loops through three clarifications has technically honored the limit.

If you want a second pair of eyes on the arm design or the labeling rubric, a free 30-minute discovery call is the cheapest way to pressure-test the plan against your volume and intent mix (Quanta Consulting).