How to Pick the Span a Disclosure Reviewer Should Cite

Agree on a vocabulary before testing anything. Three span types cover almost every disclosure finding, and each maps to a different kind of evidence.

Clause. A grammatically complete unit inside a sentence, usually marked off by a comma, a subordinating conjunction ("because," "unless," "if"), or a relative pronoun ("which," "that"). Disclosure language is long and multi-clause, and the finding usually lives in one clause: the condition, the qualifier, the parenthetical, the exception.

Sentence. The full sentence containing that clause. Use it when the clause alone reads as an assertion with no visible antecedent, or when the finding depends on a subject or object introduced earlier in the same sentence.

Character range. Two integer offsets into the document text, start and end. Use it when the finding depends on a specific token: a number, a date, a name, a percentage, a defined term. A range lets the interface highlight exactly "45 days" rather than the whole phrase around it.

Annotations attached to a specific passage rather than the whole document are generally more explainable and actionable than a single score, which is why span-level labeling is used for tasks from translation error detection to propaganda identification the LLM span annotators study.

Write down four boundary conventions: does the span include the trailing period, does it include a leading bullet or list marker, how are hyphenated terms treated, and are line breaks normalized out of the offsets. Boundary placement is the most subjective and labor-intensive annotation decision, which is why protocols fix conventions explicitly rather than leaving them to judgment Claru's annotation glossary.

Default to a clause, widen only on a named trigger, narrow to a character range only when a token is the whole finding.

1. Ask what a reviewer must see to judge the finding. Usually it is a claim plus its condition, or a claim plus its scope.
2. Select the clause containing both. Check it reads as a complete thought alone; if it starts with "which" or "because" and has no visible antecedent, widen.
3. Widen to the sentence. If the sentence holds more than two independent claims, keep the justify step at sentence but pass a character range as the precise highlight, so the interface can dim the rest.
4. Narrow to a character range when the finding turns on an exact token: a deadline, a threshold, a fee, a jurisdiction, a named party. Pad by one to three words on each side so a negation or quantifier ("not," "except," "at least") sits inside the highlight.
5. Record span type, offsets, and the offsets of the enclosing sentence, so the interface can show context without re-segmenting.

The clause-versus-sentence distinction is not cosmetic. Span labeling tools model the two differently: named entity recognition assumes non-overlapping syntactic units with clean boundaries, while span categorization handles overlapping and nested spans such as noun phrases and sentence fragments and is deliberately less sensitive to exact token boundaries Prodigy's span documentation. Disclosure findings are often the second kind: one clause can carry two obligations, and one obligation can span a parenthetical that interrupts it.

One rule saves time in practice: keep the locate span and the justify span separate. The locate span is what the highlight draws (a character range). The justify span is what a human needs to read (a clause or sentence). Pipelines that conflate them end up with noisy full-sentence highlights or incomprehensible four-word ones.

Detection quality is usually validated with an alignment sample: model results compared against human reviewers' results, discrepancies analyzed, process repeated DISCO's document review guide. That validates the decision, not the presentation. To validate the span, run a separate experiment with time as the outcome.

• Sample. 30 to 50 real findings, stratified by type: entailment gaps, missing disclosures, contradictory figures, scope-limited claims. Mix true positives and false positives roughly as your queue looks.
• Conditions. Two or three versions of the same findings, differing only in citation span. Randomize order per reviewer to blunt learning effects.
• Reviewers. Three is enough for a first read; two is a minimum. Do not let the person who wrote the span logic be the only reviewer.
• Outcome. Time from finding displayed to decision recorded, and whether the decision matches your gold judgment.
• Metric. Compare medians, not means; one reviewer losing two minutes can swamp the average. Report the share of decisions made within 30 seconds as a secondary measure.
• Repeat. Rerun whenever span rules or the document mix changes. Span quality decays silently as document types shift.

With 30 to 50 findings and three reviewers you have 90 to 150 observations, enough to detect a difference of a few seconds in median decision time, not enough for subgroup analysis. Resist drawing conclusions from eight observations of one finding type.

Expect realistic results. Studies of model and human span annotators find models achieve only moderate agreement with skilled humans and make errors at a similar rate to skilled crowdworkers, at a fraction of the cost per annotation the span annotators study. That is the bar for your span generator: not agreement with a perfect oracle, but a competent human's error rate, applied consistently.

Three patterns in the protocol data tell you most of what you need.

Fast correct decisions, slow corrections. Reviewers decide quickly but a meaningful share of decisions are later reversed. The span is complete but misleading. Typical cause: a clipped negation. The finding depends on "shall not exceed 30 days" and the highlight covers only "30 days." Fix with padding, not widening.

Slow correct decisions. The span holds the evidence but is not legible. The reviewer must read around the highlight to confirm context. Keep the narrow highlight and add the enclosing sentence as a dimmed context layer.

Fast incorrect decisions. The span is too confident: a sentence that appears to settle the question but omits a qualifier elsewhere. This is a detector problem dressed as a span problem. Widen to include the qualifier, or attach a second span. Pair timing with error rate or you will optimize into a fast, wrong queue.

Analysts reviewing benign DLP alerts routinely need context from outside the alert, because it does not say whether a user was traveling or staging data Prophet Security's analysis of DLP queues. Disclosure findings have the same shape. If the citation answers "what triggered this" but not "what makes this a problem," reviewers finish the job by hand, and the span test measures how much of that you have automated.

A generic span policy wastes what you know about the finding.

Two rules cut across all types. If the finding names a defined term, the span must include the definition or link to it. If the finding depends on a date, the span must include the date's reference point, because "within 30 days of the effective date" is a different claim from "within 30 days."

Privacy needs its own rule, because disclosures often quote personal information. Sending prompt text containing personal or health information into a generative AI tool can constitute handling under privacy law, and outputs can be inaccurate even when they read as authoritative the NSW privacy regulator's guide. One summary of prompting research reports that under a quarter of employees believed their prompts are erased, while 75% said they would use AI anyway even if every prompt were saved SoSafe's overview. If your pipeline sends document text to a model, attach a redaction class to the span record.

The most common silent failure is not span choice but span serialization. A finding cites a clause in the report, and the interface highlights nothing because offsets were computed against a different version of the text.

{
  "finding_id": "F-1042",
  "doc_sha256": "…",
  "span_type": "char_range",
  "locate": { "start": 1841, "end": 1868, "text": "not later than 45 days" },
  "justify": { "start": 1802, "end": 1904, "text": "The provider must deliver the report not later than 45 days after the end of the quarter." },
  "padding_words": 3,
  "redaction_class": "none",
  "offset_basis": "unicode_codepoints_after_nfkc_normalization"
}

Four details do the heavy lifting. The document hash pins the text version so a stale offset is detectable rather than wrong. The duplicated text lets a renderer recover by search rather than fail. The explicit offset basis (code points, UTF-16 units, or bytes) prevents the classic accent-and-emoji misalignment. The padding count documents why the range is wider than the token, so nobody tightens it later.

Version span rules the way you version prompts. Prompt descriptions that drive automated review are typically versioned on each edit, with the human-facing description and the model instruction controlled separately so changes stay traceable Ziflow's ReviewAI documentation. Treat your span policy as the same kind of artifact: a versioned, testable specification.

Building the full pipeline, from model calls to the span schema to the review interface, is the kind of project you can describe and have picked up as a build: Quanta takes your description of the app or workflow and a specialist builds it with you. If findings span text, scanned pages and disclosure images, one API key for every model type at Quanta Inference keeps the plumbing to a single integration.

• Span type recorded on every finding, with offsets and the enclosing sentence.
• Four boundary conventions written down.
• Padding rule applied so negations and operators sit inside character ranges.
• Document hash and offset basis stored with each span.
• Per-finding-type rules documented and versioned.
• Redaction class attached for privacy review.
• Span experiment run on 30 to 50 findings, timing and error rate reported together.
• Re-run scheduled whenever document mix or span rules change.