How to Set Chunk Boundaries So a Fact Survives Whole

Before you touch a splitter, find the smallest thing a user would ask about and expect a full answer to. In a support knowledge base, a help center, a policy manual or a product FAQ, that unit is almost always one question paired with its complete answer. Splitting below that unit means a retrieved chunk cannot answer anything on its own; splitting above it means the embedding blurs two or three topics together.

A practical definition you can hand to an annotator:

• Atomic unit: one question line plus every sentence, list item, table row and caveat that the answer needs in order to be true.
• Not atomic: a heading alone, a paragraph mid-answer, a table row without its column header, a footnote without its referent.

Multi-part answers are where this gets interesting. A question like "What are the refund conditions for annual plans?" may answer with a duration, a percentage, an approval step and two exclusions. All of that belongs to one unit. If you split the exclusions into a separate chunk, the retrieved evidence states a refund is available without stating when it is not, and the generator can answer with confidence in the wrong direction.

Keep the question line attached to the answer in the stored text, not just in metadata. When a chunk contains its own question, its embedding carries both the user's phrasing and the answer's content, which improves matching on the exact words users type.

Once the unit is defined, the splitter's job is simple: cut immediately before each question, and never anywhere else except where a single unit exceeds your maximum chunk size. That turns the problem from "where do I cut text" into "how do I recognize a question start."

Typical question-start signals in documentation:

When your source has a structured FAQ format, parse the question and answer fields directly instead of splitting on punctuation. That removes the guesswork. Where the content is prose with embedded questions, use the heading and question-mark signals to place boundaries and then walk backward to attach any lead-in sentence that the question depends on, such as "The following rules apply to all annual plans."

Here is a minimal splitter that respects those rules. It cuts only at question markers and keeps each unit whole:

import re

QUESTION_START = re.compile(
    r"(?=^#{2,6}\s.*\?\s*$)|(?=^\*\*.+\?\*\*\s*$)|(?=^Q:\s)",
    re.MULTILINE,
)

def split_on_questions(text: str, max_chars: int = 1800) -> list[str]:
    units = [u.strip() for u in QUESTION_START.split(text) if u.strip()]
    chunks = []
    for unit in units:
        if len(unit) <= max_chars:
            chunks.append(unit)
            continue
        # Oversized unit: split on sentence ends, then repack,
        # never leaving a sentence fragment in a chunk.
        sentences = re.split(r"(?<=[.!?])\s+(?=[A-Z0-9])", unit)
        current = ""
        for s in sentences:
            if len(current) + len(s) + 1 > max_chars and current:
                chunks.append(current.strip())
                current = s
            else:
                current = f"{current} {s}".strip()
        if current:
            chunks.append(current)
    return chunks

Two details matter more than the code. First, the sentence split uses a lookbehind on terminal punctuation and a lookahead on an uppercase letter or digit, which reduces (but does not eliminate) cuts inside abbreviations. Second, the oversized-unit path keeps at least one full sentence per chunk, so even a very long answer never returns a bare fragment.

The sentence splitter above still has a weakness: a sentence can be longer than your limit, and some "sentences" contain decimal numbers, version-like tokens or abbreviations that look like sentence ends. Add a guard pass that forbids cuts inside protected spans.

A simple enforcement rule: after splitting, reject any chunk whose first or last token matches a number, currency symbol, month name, unit word, or dangling connective (and, or, unless, except, if). Move those boundary tokens to the neighboring chunk and re-check. This is mechanical and catches most half-facts before they ever reach an embedding.

For conditions specifically, prefer a split that keeps the whole if ... then ... construction in one unit even if it runs a little over your target length. Chunk size is a soft budget. A condition split across two chunks is a hard error. The Meilisearch guide makes the same point in its evaluation advice: chunks should be self-contained facts that are not cut off in a way that changes them per its context preservation measure.

Sliding-window overlap is the default fix for boundary loss, and it is the wrong one here. Repeating the last 50 tokens of an answer into the next chunk duplicates half a fact if the boundary fell inside it, and it makes the second chunk's embedding drift toward content it does not own. Meilisearch lists overlapping as a valid umbrella strategy but pairs it with the caveat that the chunk must still hold one coherent fact per the same guide.

For question-boundary chunking, repeat the question line at the top of every continuation chunk of a long answer. The question is the retrieval key, so carrying it forward keeps continuation chunks matchable without duplicating answer content. If an answer spans three chunks, all three start with the same question, and the numbers appear exactly once each.

Pair this with metadata that lets you reassemble rather than duplicate:

• doc_id and unit_id identify the question-answer pair.
• part and part_count show whether the unit was split and where this chunk sits.
• has_question_line records whether the chunk starts with the question.
• protected_spans can cache the character offsets of numbers, dates and conditions you refused to cut.

At query time, if two chunks share a unit_id, the generator can see they are parts of one answer. That is cheaper and more accurate than hoping overlap covered the gap.

You cannot know the boundaries are right by reading the splitter. You need a small query set with known-answer checks. Build 20 to 50 real questions your users ask, and for each one record the exact strings the answer must contain: the number, the date range, the condition. Then run retrieval and check the text, not the model's answer.

A validation set in that shape looks like this. The must_contain list is the automated check; the must_not_lead_with list catches fragments:

cases = [
    {
        "query": "Can an enterprise customer cancel after renewal and get money back?",
        "must_contain": ["30 days", "prorated", "annual plans"],
        "must_not_lead_with": ["percent", "days of"],
    },
    {
        "query": "How long does a refund take once approved?",
        "must_contain": ["5 to 10 business days"],
        "must_not_lead_with": ["business"],
    },
]

Score each case on three counts. Hit rate: did any retrieved chunk contain all required strings? Intactness: did the top-ranked chunk contain them without the answer being split across that chunk and its neighbor? Grounding: does the generated answer cite a chunk that actually holds the fact? This mirrors the evaluation dimensions used in legal QA chunking research, which measured answer correctness, answer groundedness and quality of retrieved chunks against a constructed question-answer set per the IEEE conference paper. The paper also compared naive token-level, hierarchical sentence-level and recursive chunking on the same corpus, which is the right experimental shape: hold the model constant and change only the boundaries.

A worked example with numbers helps set expectations. Suppose you have 4,000 FAQ units averaging 600 characters, and a fixed-size splitter at 1,800 characters produces roughly 1,700 chunks. If 15 percent of boundaries land inside a unit, that is about 255 chunks with a cut fact. At a top-5 retrieval budget, a query hitting a damaged unit has a real chance of pulling the truncated half, because truncated chunks often score well on the question terms. Moving to question boundaries removes those 255 splits at the cost of some chunks being shorter or longer than target, which is a good trade for a fact-bearing corpus.

Fix failures at the unit, not the chunk. If a case fails intactness, the answer is almost always a multi-part answer that should have been one unit, or a condition that was written two paragraphs below its rule. You can also raise the retrieval count, but a larger candidate pool does not repair a split fact. It just gives the model more fragments to reconcile, and the arXiv work on question decomposition shows why: when the needed evidence is distributed, retrieval improves only if each piece is self-contained and then reranked, not if the pieces are halves of the same statement per the question decomposition paper.

Question-boundary chunking sits in preprocessing, before embedding, and it interacts with two later stages. First, embeddings: a chunk that leads with its question has a strong lexical and semantic match to user phrasings, which matters because chunk quality directly affects retrieval accuracy per IBM's RAG overview. Second, reranking: once you retrieve a candidate pool, a reranker can prefer chunks where the full fact appears, but it cannot create the missing half. Better boundaries reduce the reranker's workload.

If your team is still assembling the pipeline itself, AWS's primer on retrieval-augmented generation walks through the retrieve-then-generate loop and where the knowledge base step sits per AWS. For a hands-on local build that shows text splitting in context, Daniel Bourke's from-scratch RAG tutorial covers preprocessing, chunking, embeddings and generation in one sequence per the tutorial. Watching a splitter misbehave on a real PDF is the fastest way to internalize why fixed windows fail.

One caution that applies throughout: do not carry over prompt-engineering habits that were tuned for plain chatbots. Aaron Tay's analysis of RAG prompting argues that techniques tested on chat models do not automatically transfer to retrieval systems, because the model is reasoning over retrieved context rather than free generation per his analysis. The same logic applies to chunking: a trick that made a chatbot answer better says nothing about where your evidence boundaries belong.

For teams that want one API key across embeddings and generation so chunking experiments do not require rewiring providers, Quanta Inference exposes text, image, video, audio and embedding models through one OpenAI-compatible, prepaid endpoint Quanta Inference. That keeps the variable under test as the boundary rule, not the vendor.