Tool Results Are Untrusted: Sanitizing Before Context

One global cap is the wrong instrument. A 2,000 token search result is generous, a 2,000 token file read is cramped, and a 2,000 token database row is usually a query bug. Set caps by what the tool is for and what complete means for it.

Per-tool configuration is already standard in coding harnesses. A public harness feature request documents the shape: a token budget per tool, an optional line limit, a truncation strategy, and a spill threshold above which full output goes to disk with a retrieval hint (pydantic-ai-harness issue). It also records what established harnesses do: Codex truncates at roughly 10 KiB or 256 lines with a head plus tail strategy, and one per-tool-type approach returns full file content only up to about 4 KB before switching to line ranges, top three search matches, and the last 1 KB for commands. Treat those as reference points read from that issue, then calibrate against your own traces.

Two rules make caps work. First, the cap names a unit (bytes or tokens) and a measurement point, so string length never passes as a token budget; Haystack's compaction docs separate measurement from strategy the same way, with a token counter estimating size before anything is sent to a model (Haystack). Second, the host application keeps the full untruncated result even when the model sees only the trimmed version, which is what lets you debug a turn without re-running the tool.

The cost side matters too. Agent loops that append every tool output to history and resend it on each call pay repeatedly for the same bytes, so cumulative input cost grows faster than the number of steps (Augment Code). A cap at receipt is the cheapest point in the pipeline to stop that.

A cap answers how much. Validation answers is this the thing I asked for. Both run before the text enters the conversation.

1. Transport check. Did the call succeed, and is the payload the type the connector expects? A 500 page is text; it is not a search result.
2. Shape check. Validate against the tool's declared schema: required fields, types, array lengths, no unexpected top-level keys. Reject on failure rather than coercing.
3. Cap check. Measure in the declared unit against the per-tool cap. Over the cap becomes its own failure code, not a warning.
4. Provenance check. Attach source id, the tool's timestamp, the parameters used, and a hash of the raw payload.

Provenance is where evaluation practice and runtime practice meet. A RAG testing strategy tracks what entered the index, which filters applied, which chunks ranked, what context reached the model, and whether the answer is supported by it (qaskills.sh). Those are the same fields you need to decide whether to keep a tool result. If the gate does not emit them, you cannot tell later whether a bad answer came from a bad tool or a bad gate.

For text that passes, two small hygiene steps pay for themselves: strip control characters and invisible Unicode, which OWASP lists as an obfuscation vector (OWASP), and wrap the kept text in a delimiter naming its origin, so traces and prompts make the boundary visible. Delimiters do not stop injection; they make the boundary auditable.

def admit(tool_name, raw, schema, caps):
    try:
        payload = json.loads(raw)
    except ValueError:
        return Reject("not_json")
    if not schema.validate(payload):
        return Reject("schema_mismatch")
    size = count_tokens(raw) if caps.unit == "tokens" else len(raw.encode())
    if size > caps.max:
        return Oversize(size=size, cap=caps.max, source_id=payload.get("id"))
    text = clean_text(raw)  # strip control chars and invisible Unicode
    return Keep(text=text, source_id=payload.get("id"), raw_hash=sha256(raw))

Note what the function refuses to do: it never returns a silently trimmed string. A result is kept whole, or it is labelled oversize and handed to a policy that makes an explicit choice. Silent trimming is the failure this design exists to prevent.

Once a result fails the cap, the turn needs a decision. Three moves exist, and each hides a different failure.

Drop is honest but blunt. Its value is telling the model the truth: nothing usable came back. That beats handing over half a document with no marker. Arize's account of an agent built over trace data describes what naive truncation does instead: capping every input at a fixed character count traded one failure for another, because the agent lost continuity and stopped behaving coherently (Arize). That post also reports a negative result worth repeating: LLM summarization of history was tried and abandoned.

Summarize is a good default only when you keep the source id and can re-fetch. A fixed budget (a set token count per summary, with the original stored outside context) keeps it predictable. Haystack is direct about the trade: compaction is lossy, and what survives depends on the strategy, with summarization adding model cost and latency while remaining lossy (Haystack).

Re-call is the strongest move and the easiest to abuse. It works when the failure is really a parameter problem: a select star on a wide table, a search with no date filter, a file read with no line range. It fails when the tool is wrong, and then it hides the bug behind a longer trajectory. Cap re-calls per tool per turn and log each one; the same tool called twice with identical arguments is a bug, not a retry.

A worked example

A research agent is asked what last quarter's revenue looked like. Search returns 40 hits. A file read returns an earnings PDF as 180,000 tokens of extracted text against a 6,000 token cap. A database tool returns 12,000 rows against a 200 row cap.

• Search: kept, top 8 hits, scores recorded for all 40. No policy fires.
• File read: oversize. Summarize into a 1,200 token budget, keep document id and page numbers, store the raw text outside context. The model sees the revenue figure sits on page 12 and re-reads pages 11 to 13, which passes.
• Database: oversize. Re-call with a quarter grouping and a projected column list. The narrow result is 4 rows and passes.

Now the same turn with defaults, where most teams start. The file read is truncated at 6,000 tokens, landing mid-table, so the revenue row is cut and the model reports growth from a partially visible table. The database result is truncated at 200 rows, a silent sample, so the model totals the first 200 of 12,000 and is wrong by whatever share those rows represent. Neither failure raises an error. Both produce fluent answers.

Decide this before writing code: oversize is a state change for the turn, not a warning attached to a message.

• The turn continues, but the tool's contribution is marked partial or absent, and the model is told which.
• The gate increments a counter per tool per session. A tool that fires often is miscapped or misdesigned, and the counter is how you find out.
• The turn budget is checked. If kept text plus new summary plus remaining tool definitions exceeds the threshold, compaction runs. Haystack's example starts at 70 percent of the window and reduces to about 40 percent, a reasonable starting pair read from those docs, not a recommendation for your workload (Haystack).
• If the turn already dropped or summarized and still cannot progress, stop and return a partial answer with the reason. An agent that says it could not read the source in full is more useful than one that guesses.

For tools whose output must stay complete, the pointer pattern is the alternative: the agent passes a reference to stored data and the tool resolves it. One study reports that approach consuming roughly seven times fewer tokens than pasting the output into context while preserving it fully (arXiv 2511.22729). That is a strong result for one domain, and it requires your tools to accept references. Where they can, it removes the drop-versus-summarize dilemma for large, indivisible outputs.

A related direction is models that manage their own context. Context Language Models treat context as a file the model edits, reporting higher accuracy with fewer FLOPs than external harness strategies on several benchmarks (arXiv 2609.37725). Worth watching, but until your model does this natively, the gate sits in your harness and you own the policy.

A gate you cannot measure is a guess. Three counters and one log line cover most of it:

• Reject rate per tool and per failure code (not_json, schema_mismatch, oversize).
• Policy mix per tool: the share of oversize events dropped, summarized and re-called.
• Recovered rate: the share of turns reaching a correct final answer after an oversize event, measured on a small labeled set.

The last one is the decision rule. If summarized events recover as often as clean ones, the summary budget is fine. If dropped events recover far less often than summarized ones for a tool, you are dropping evidence you need, and the cap or policy is wrong. Contextual precision and recall give the same read on retrieval: whether the right evidence made it into what the model saw (Confident AI). Evidently's guide makes the same split between testing retrieval and testing generation, because one end-to-end score cannot say which side broke (Evidently AI).

Track summarize latency too. Summarizing adds a model call; if a turn makes 12 tool calls and half summarize, you doubled the calls. Since summaries can be cached by raw payload hash, a repeated identical result costs nothing the second time.

Put it in the tool wrapper, not the prompt and not the model client. The wrapper knows the tool's schema, its cap and its narrowing parameters, and it is the only place that sees the raw payload before the conversation record does. Haystack's split of trigger (when to compact), strategy (how to rewrite) and measurement (how big things are) maps directly onto tool results (Haystack).

The broader frame is context engineering, and the vocabulary of write, select, compress and isolate covers it (LangChain). The gate is select plus compress at the point of entry. Isolation is the other lever: run high-volume tools inside a sub-agent whose context is discarded and whose only output is a short structured finding, so the oversized result never touches the main turn.