Using Retrieval to Answer Support Questions from Your Own Pages

The chunk is the unit your system retrieves and cites, so chunk quality sets the ceiling on answer quality. Two failure modes dominate. Chunks too small lose context: "Click Save" tells the reader nothing without the surrounding screen. Chunks too large dilute the embedding, because a 900-token chunk covering four topics produces a vector that is an average of all four and matches none of them well.

A practical default for help content is a few sentences per chunk with modest overlap, and a commonly cited sweet spot in production systems is roughly 400 to 512 tokens with a small overlap between neighbors (Adnan Masood on chunking strategies). For support pages you can often go shorter, because a step or a policy clause is naturally short. The overlap exists so a sentence that straddles a boundary appears in at least one chunk intact.

Structure should drive the split before length does. Help articles already have headings, numbered steps, and tables, and cutting on those boundaries produces chunks that are semantically coherent. Research on hierarchical segmentation finds that chunking along a document's real structure, rather than fixed character counts, retrieves more relevant context for complex questions (arXiv on hierarchical text segmentation chunking). A clinical-domain comparison of chunking methods reached a similar conclusion, that the choice of chunking strategy measurably changes retrieval and downstream answer quality (PMC chunking evaluation).

Keep tables intact and treat them as single chunks, even if they exceed your token target. A pruned table is worse than no table. When you do split, prepend a short breadcrumb to each chunk before embedding: article title plus heading. That single line gives every chunk the context that "it depends" or "the third option" otherwise lacks.

Once you have chunks, embed each one with a text embedding model and write the vector plus metadata into a search index. The embedding model and the generation model do not need to be from the same vendor; what matters is that you use the same embedding model for indexing and for querying. Mixing them silently wrecks similarity scores.

The metadata you store per chunk is what turns a retrieved passage into a citation. At minimum, keep the article title, the URL, the heading path, the chunk's ordinal position in the article, and, if the article has anchors, the fragment ID. Citation-aware systems store these anchors separately from the chunk text so the text stays clean while the pointers stay precise; adding this metadata typically costs on the order of 10 to 15 percent extra storage (Tensorlake).

A chunk record looks roughly like this:

{
  "id": "kb-412-downgrade-3",
  "article_title": "Changing your plan",
  "url": "https://help.example.com/plans/change",
  "anchor": "#proration",
  "heading_path": ["Plans", "Changing your plan", "Proration"],
  "position": 3,
  "text": "When you downgrade mid-cycle, the unused portion of your current plan is credited to your next invoice. The change takes effect at the start of your next billing period.",
  "embedding": [0.0123, -0.0456, "..."]
}

When you choose the embedding model, match it to your language coverage and query length. Support questions are short, often a sentence or less, so a model trained to align short queries with longer passages will rank better than one that assumes symmetric inputs. Re-embedding the whole corpus is the expensive operation, so it is worth testing on a few hundred real questions before you commit to a model.

At query time the flow is: embed the question with the same model, run a similarity search for the top 3 to 5 chunks, and pass those chunks to the generation step with their metadata attached. Ask the model to answer only from the provided passages and to name the source it used. Returning the passages themselves, not just a generated summary, is the design choice that keeps trust high; the reader sees the evidence.

The confidence threshold is the most important number in the system and the one most teams skip. Each retrieved chunk carries a similarity score. Set a floor, and if the best score is below it, do not answer. Hand the question to a human with the retrieved candidates attached so the agent starts with context. Calibrate the floor empirically on real questions rather than guessing it. Pull a few hundred historical support questions, score the best match for each, and sort by score. Read the bottom of the distribution first: those are the questions your index cannot answer well, and that is where your threshold belongs.

The same principle applies when you generate text: attribution is a separate task from generation, and asking one model to invent citation markers inline is unreliable, with inline accuracy for strong models landing in a wide band rather than at 100 percent (Zero Entropy on citation extraction). The safer design is to generate the answer from a small set of passages and attach the passage metadata directly, so the citation is copied, not composed.

Suppose your help center has 600 articles averaging 1,200 words, or roughly 1,600 tokens each. That is about 960,000 tokens of source text. At 400 tokens per chunk with 50-token overlap, the effective stride is 350 tokens, so you get roughly 2,750 chunks. Call it 2,800 after rounding and table handling. Indexing 2,800 vectors is trivial; a single batch job does it.

Now the query side. Say support handles 40,000 questions a month. Each question triggers one embedding call, one vector search, and, for the questions that clear the threshold, one generation call over three passages of about 400 tokens each, so roughly 1,200 tokens of context plus the question. If 70 percent of questions clear the threshold, that is 28,000 generations and 12,000 handoffs a month. The 12,000 handoffs are not a failure of the system; they are the system correctly declining to guess, and the agent receives the top candidates to speed up resolution.

Two numbers to watch after launch. First, the handoff rate: if it climbs well above your baseline, either your threshold is too high or your index is missing content (common with internal-only articles that were never published to the help site). Second, citation click-through: if readers rarely click the cited passage, they may be trusting the generated text and skipping verification, which is fine when answers are right and dangerous when they are not. Track both.

If you would rather not assemble embedding, search, and generation yourself, Quanta Inference puts text, embedding, and other model types behind one OpenAI-compatible API key, which lets you swap the embedding model without rewriting your index code (https://quantaintelligence.ai/inference).

Treat the pipeline as a product with its own monitoring. Log every query, the top match score, whether it cleared the threshold, and, when a human took over, what the agent actually did to resolve it. Those logs are your improvement backlog. Recurring low-score questions point to missing articles. Recurring high-score questions that still get escalated point to passages that retrieve well but answer badly, usually because the passage is a caveat rather than the fix.

A few operational habits keep quality up. Re-index when articles change: stale vectors return passages that no longer match your current product. Keep a small evaluation set of real questions with known correct passages and rerun it whenever you change chunking or the embedding model, because both changes move scores and can shift your threshold's meaning. And version your threshold alongside your model choice; a number tuned on one embedding model does not transfer cleanly to another.

When the assistant does escalate, make the handoff generous. Pass the question, the top three candidate chunks with their scores, and a link to each source. An agent who opens a ticket already looking at the two closest articles resolves it faster, and you get a clean feedback loop: the article the agent used is a strong signal for what the retrieval should have ranked first. That loop, not the model, is what makes the assistant improve over time.