How to Map the Questions Your AI Front Desk Will Get

A good question map draws from every channel where customers already express intent, because each channel biases the sample differently.

Pull a window long enough to catch seasonal and billing-cycle peaks: one full quarter is usually enough for a first pass, and for a small operation 500 to 2,000 conversations gives you a useful sample. Keep the raw export intact, including timestamps and channel, so you can re-sample later.

Three details matter more than volume. First, take the customer's first message verbatim, not the agent's summary; the summary has already translated the question into your vocabulary, which is exactly the bias you are trying to remove. Second, capture zero-result searches separately, since every one of them is a documented content gap (Zendesk). Third, capture questions that ended in escalation even though an article existed: those are structure or findability failures, and they belong on the map with a different label.

Raw exports are noisy. Cleaning is mostly mechanical, and skipping it is the most common reason an intent map ends up with 400 categories that nobody uses.

1. Strip greetings, signatures, ticket IDs and account numbers.
2. Remove non-questions: thank-you notes, satisfaction surveys, internal comments.
3. Separate questions by topic: billing, account access, shipping, returns, how-to, technical troubleshooting, policy, and "talk to a human".
4. Split compound messages. A message that asks about a refund and a shipping delay is two questions, and your agent will answer both badly if you tag it once. Splitting subjects into separate chunks measurably improves retrieval, because a chunk that covers two subjects returns weaker similarity scores than two focused chunks (Meilisearch).
5. Normalize wording without losing it. Map to a canonical form ("cancel subscription", "end my plan", "stop billing") but keep the original phrasings attached as aliases, because they are your future test cases.

Intent recognition, the process of identifying the goal behind a message rather than matching keywords, is what makes grouping possible: it distinguishes "I want to pay my bill" from "why is my bill so high" (IrisAgent). A useful rule of thumb from that same practitioner guide is to start with 30 to 40 intents and expand toward 60 or 80 as the map matures, organizing them into categories such as account management or billing, and always including a fallback intent for messages that match nothing (IrisAgent).

Worked example. Suppose you pull 1,200 first messages. Cleaning leaves 900 questions. You split 140 compound messages, giving 1,040 question units. Grouping at a sensible granularity yields 46 intents: 12 account and billing, 11 shipping and returns, 9 product how-to, 8 policy, 4 integrations, 2 "talk to a human". Forty-six is a healthy first map. If you land at 150, your intents are too narrow (you are clustering synonyms); if you land at 12, they are too broad and your answers will hedge.

Frequency alone misleads. A question asked 200 times whose worst outcome is a mild annoyance matters less than one asked 30 times that, answered wrongly, produces a refund dispute or a compliance problem. Rank intents with an expected-loss score:

score = volume share × (time saved per correct answer + cost of a wrong answer)

For each intent, record four numbers: how many times it appeared in the sample, the share of the total, an estimate of the minutes a correct automated answer saves (or the ticket it avoids), and a severity weight for a wrong answer on a 1 to 5 scale (1 = cosmetic, 5 = financial, legal or safety impact).

Worked example, with a $40 blended cost per human-handled contact:

The top row is the highest volume, but refund eligibility scores nearly as high despite half the volume, because a wrong answer there has real money attached. Shipping the password intent first and the refund intent second is defensible; shipping password first and never getting to refunds is not.

Two guardrails keep the ranking honest. Exclude intents the AI should never answer autonomously (legal threats, account security incidents, anything requiring identity verification) and route them to a human with context, which is the pattern many support platforms follow when confidence is low (IrisAgent). And treat the severity weight as a policy decision owned by someone accountable for it, not by whoever ran the export.

For each top-ranked intent, write one entry that answers the whole intent, not one phrasing of it. Generative AI splits content into chunks, stores each as a vector, and generates replies from the best-matching chunks plus the article title, so the quality of the chunk is the quality of the answer (Zendesk help).

Structure each entry so a machine can retrieve and use it:

• A title that repeats the customer's phrasing, since the title is used in reply generation.
• A direct answer in the first sentence, no preamble.
• Complete instructions as a numbered list, with each alternative solution presented separately rather than as nested sub-steps.
• The customer's own vocabulary, with acronyms spelled out on first use and vague pronouns replaced by nouns (Zendesk help).
• One topic per article, with redundant or contradictory older versions removed.

The self-contained rule deserves emphasis. Zendesk's guidance is to avoid linking out and to include everything the user needs in the topic, because a passage that says "see the other article" gives retrieval nothing to answer from (Zendesk help). HelpDocs describes the same failure from the other side: an article a person would eventually piece together from three sections is an article an AI will answer badly (HelpDocs).

Chunking choices belong here too. NVIDIA's experiments across several document sets found page-level chunking produced the highest average end-to-end accuracy at 0.648 with the lowest standard deviation of 0.107, while very small chunks of 128 tokens and very large chunks of 2,048 tokens generally underperformed medium sizes, and optimal size varied by document type, with one financial benchmark favoring 1,024 tokens and another 512 (NVIDIA). The decision rule that follows: write short, single-topic entries, then measure retrieval on your own questions rather than picking a chunk size from a blog post, using precision and recall against known-answer chunks (Meilisearch).

A question map doubles as an evaluation set. Save the original customer phrasings, with their intent labels and a reference answer, and after every content change ask the agent each one and grade the result as correct, hedged, or wrong. HelpDocs recommends running five real customer questions against a connected agent as a minimum smoke test: the usual outcome is two good answers, two hedges, and one confident invention (HelpDocs). A mapped set of 50 to 100 questions turns that anecdote into a trend line.

Track three numbers per intent: share of test questions answered correctly, hedge rate, and escalation rate. A high hedge rate usually means the answer exists but is buried or split across articles; a high escalation rate on a high-volume intent means the content is missing or the agent lacks the tool to act. Measure deflection alongside, since self-service quality shows up as fewer tickets, not just happier transcripts (Crisp). Zendesk's list of resolution-focused metrics (self-service ratio, ticket deflection, AI resolution rate, search success, content health) is a reasonable dashboard skeleton (Zendesk).

Keep the map alive with a monthly routine: pull a fresh sample of new questions, tag them against the existing intents, and add a new intent only when it appears at least five times across two consecutive samples. Review zero-result searches every week, because they are the cheapest signal of a gap you can close within hours. Audit content quarterly for contradictions and stale policies, since duplicate, conflicting versions are one of the three main causes of grounded-AI failure (HelpDocs).

Finally, pick the model layer after the map, not before. Use the strongest available model to establish a baseline, set up evals, then trade down to smaller, faster models where accuracy holds, which keeps cost per conversation under control as volume grows (OpenAI). If you would rather not run that plumbing yourself, Quanta Inference offers text, image, video, audio and embedding models through one OpenAI-compatible API key, priced in US dollars and prepaid, which makes swapping a model during evaluation a one-line change (Quanta Inference).

Put the whole loop on a calendar so it survives contact with busy weeks:

1. Week one: export last month's questions from every channel in the harvest table.
2. Week one: clean, split compound messages, and tag against the existing intent list.
3. Week two: update volume counts, recalculate scores, and pick the top three content tasks.
4. Week two: run the 50 to 100 question test set and log correct, hedged and wrong.
5. Week three: rewrite or create the entries, one topic per article, self-contained.
6. Week four: re-run the test set, review zero-result searches, and retire contradictory articles.

The value compounds. Each month the intent list gets sharper, the test set gets harder, and the content gaps get narrower, which is the only reliable way to raise the ceiling of a grounded assistant (HelpDocs).