AI agents

Splitting a Memory Store by Trust: Read-Only, Draft, Quarantined

Antonio Evans9 min read

A cutaway diagram of a memory store split into three vertical columns.
Key points

Key points

  1. 1Split memory into three tables with different permissions: read-only answers, write-only drafts, and quarantined rows.
  2. 2Every unverified write lands in draft, and the answering path cannot read draft rows at all.
  3. 3A draft is promoted to read-only only when a second independent source confirms the same fact.
  4. 4A read-only row that a later check contradicts moves to quarantine rather than being deleted silently.

Why permission beats policy in agent memory

Most agent memory failures are not retrieval failures. They are trust failures. A memory system that lets the same code path write and read a fact gives an unverified claim the same standing as a checked one, and no amount of prompt wording fixes that. The moment a claim is in the queryable store, some future turn will surface it as context, and the model has no way to tell a checked fact from a scraped one.

Research on memory poisoning makes the mechanism concrete. The GhostWriter attack reported near-universal injection rates of roughly 98% and an average activation rate of roughly 60% against state-of-the-art agents, and the authors attribute the success to a lack of security-focused memory governance (the memory poisoning paper). A separate analysis describes the same pattern as temporal decoupling: the injection happens now, the damage happens later, triggered by an unrelated query (Schneider's write-up).

That is the case for a structural fix. If the answering path physically cannot read unverified rows, an injected claim can be stored without ever becoming an answer. The agent can still write what it hears. It just cannot treat what it heard as knowledge until something else agrees.

The distinction matters because the obvious defenses do not cover this. Input moderation catches known-bad patterns but not fluent, plausible falsehoods. TTLs remove old data but a poison planted a minute ago is still fresh. Chunk-level quarantine flags a suspicious passage but does nothing about a fact that arrived clean, looks reasonable, and is simply wrong.

The three tables and what each one is for

The schema is three tables with one shared shape and three different access rules.

TableAgent can writeAgent can read for answersPurpose
memory_readonlyNo (worker only)YesVerified facts the answer path may cite
memory_draftYesNoUnverified writes awaiting a second source
memory_quarantineNoNoContradicted or untrusted rows, kept for audit

The permission model is the whole point. The answering path holds a role that has SELECT on memory_readonly and nothing else. The write path holds a role that can INSERT into memory_draft and INSERT into memory_quarantine, and can UPDATE only the promotion bookkeeping. A single generic remember(text) tool breaks all of this, which is why Zep's defense guidance recommends one application-owned write path with typed operations rather than an open text sink.

Here is a minimal schema that enforces the split in the database rather than in application code:

CREATE TABLE memory_readonly (
  id            BIGSERIAL PRIMARY KEY,
  subject       TEXT NOT NULL,
  field         TEXT NOT NULL,
  value         TEXT NOT NULL,
  source_id     TEXT NOT NULL,
  accepted_at   TIMESTAMPTZ NOT NULL DEFAULT now(),
  UNIQUE (subject, field, value, source_id)
);

CREATE TABLE memory_draft (
  id            BIGSERIAL PRIMARY KEY,
  subject       TEXT NOT NULL,
  field         TEXT NOT NULL,
  value         TEXT NOT NULL,
  source_id     TEXT NOT NULL,
  source_kind   TEXT NOT NULL,
  observed_at   TIMESTAMPTZ NOT NULL,
  accepted_at   TIMESTAMPTZ NOT NULL DEFAULT now(),
  status        TEXT NOT NULL DEFAULT 'pending'
);

CREATE TABLE memory_quarantine (
  id            BIGSERIAL PRIMARY KEY,
  original_id   BIGINT NOT NULL,
  original_tbl  TEXT NOT NULL,
  subject       TEXT NOT NULL,
  field         TEXT NOT NULL,
  value         TEXT NOT NULL,
  reason        TEXT NOT NULL,
  moved_at      TIMESTAMPTZ NOT NULL DEFAULT now()
);

REVOKE ALL ON memory_draft FROM answer_role;
REVOKE ALL ON memory_quarantine FROM answer_role;
GRANT SELECT ON memory_readonly TO answer_role;

Two design details carry real weight. First, source_id is not the agent session. It is the identifier of the human, document, API response or button press that supplied the value. Second, observed_at and accepted_at are separate columns on purpose. An attacker who controls a timestamp in a document can claim an event happened yesterday; they cannot claim the application accepted the record yesterday. Zep notes that keeping these two dates apart removes recency as a lever for poisoning (Zep).

The write path: everything unverified lands in draft

Every write from an untrusted surface goes to memory_draft. Untrusted means the value's truth lives somewhere the agent does not control: a user message, an email body, a fetched web page, a support ticket, an imported document, a tool's free-text output. The agent may still propose the write. Application code decides what shape it takes.

The safe pattern is a typed operation with a fixed field list, not free text:

ALLOWED_FIELDS = {"timezone", "billing_country", "preferred_language", "plan_tier"}

def record_claim(subject, field, value, source_id, source_kind, observed_at):
    if field not in ALLOWED_FIELDS:
        return {"status": "rejected", "reason": "unknown field"}
    return db.insert_draft(subject, field, value, source_id, source_kind, observed_at)

This is the rule captured in a Rasa walkthrough of who should write agent memory: if the customer chooses a value, the model may record it from a fixed list; if your own systems know it, only a tool writes it (the Rasa episode). Eligibility, account status and refund amounts fall in the second bucket. They should never pass through draft at all, because their truth is authoritative and a single write from the owning system is enough.

Draft rows are visible to three things and nothing else: the promotion worker, an audit console, and the user-facing transcript if you choose to show "I noted that, but I'm not using it yet." The answering path does not see them. A draft row can be retrieved by similarity search for the promotion worker's benefit, and still cannot enter a prompt that composes an answer.

Rate limits belong here too. Per-source limits on draft writes cut flooding and fake corroboration, where one attacker submits the same claim many times to look like several sources (Zep).

The promotion rule: a second independent source, not a second mention

This is the decision rule most memory systems get wrong. Repetition is not confirmation. Ten messages from one account remain one source, and a summary of a user's content is still that user's content. The GovMem work frames this precisely: repeated observations across agents are not necessarily independent evidence, because the same claim may be copied from a shared source or induced by a shared prompt (When Not to Write Memory).

A draft is promoted when, and only when, all of these hold:

  1. A draft row exists for (subject, field, value).
  2. A second row exists for the same (subject, field, value) whose source_id differs.
  3. Those two source_id values do not share a lineage ancestor. If one was derived from the other through summarization, extraction or a tool echo, they count as one source.
  4. No contradicting row exists in memory_readonly for that (subject, field) with a different value, unless that row has itself moved to quarantine.
  5. The field is on the promotable list. High-impact fields such as payment destination, health information or access level require human review regardless of corroboration.

Condition three is where authority laundering gets caught. When a model restates a claim, the restatement looks like a new source but is not (Zep). Storing a parent_source_id on every derived row makes independence checkable rather than assumed.

A worked example with numbers. Suppose the promotion worker runs every ten minutes over a draft table holding 4,000 pending rows from the last day. Of those, 3,100 have exactly one source and stay in draft. Another 700 have two rows with distinct source_id values, but 540 of those pairs share a lineage ancestor (the agent summarized an email into memory, then a later turn re-extracted the same claim), so they stay in draft. Of the remaining 160, 25 match a contradictory verified row and go to quarantine. The other 135 promote. That leaves roughly 3.4% of daily writes reaching the answer path, which is the intended funnel: cheap to write, expensive to answer from.

The review burden is real and should be counted. In GovMem's controlled stress tests, dependency-aware promotion cut false promotion from 0.597 to 0.040 while holding recall at 0.960, at an explicit cost in human review (the GovMem paper). On a project-internal labeled subset the method was far more conservative, with a 0.692 review burden and 0.448 direct recall. Plan for a queue with a person on it, or accept that the draft table will grow.

Demotion to quarantine, and why deletion is the wrong move

Contradictions happen after promotion. A fact can be true when verified and false three months later: a plan changes, an address moves, a policy is revised. The fix is not to delete the row, because deletion loses the record of what the agent used to believe and makes root-cause analysis impossible when a bad answer shows up later.

Demotion runs the same check in reverse. When any verification pass finds a read-only row contradicted by a newer, independent, authoritative source, the row moves to memory_quarantine with a reason and a pointer to the contradiction. The answer path loses it immediately, because the row is no longer in the queryable table. The audit trail keeps it.

Keep a tombstone for the rejected value. The idea of negative memory, which records a known-wrong value so it cannot quietly return through extraction, is one of the reusable patterns catalogued in the Agent Memory Atlas (the atlas). Without a tombstone, the same false claim gets extracted again next week, promoted again if the corroboration conditions happen to be met, and the cycle repeats.

This is different from TTL expiration, which drops rows on a clock and knows nothing about truth. A pure TTL policy will eventually drop a fact that was both old and still true, and an allergy or a billing structure is exactly that kind of fact (mem0 on eviction). TTL is still the right tool for legally constrained retention, and MongoDB's agent memory docs implement it as a per-user setting with a 30-day default on episodic logs, changeable at runtime (the TTL how-to). Keep TTL for compliance. Use trust tiers for correctness.

MechanismWhat it removesCatches a planted falsehoodKeeps true old facts
TTLAnything older than N daysNoNo
LRU / decayAnything not recently usedNoUsually not
Chunk quarantineA suspicious passage at ingestSometimesYes
Trust tiersUnverified and contradicted rowsYes, on the answer pathYes

The four mechanisms compose rather than compete. TTL handles retention policy. Recency scoring handles interference. Chunk quarantine handles obviously bad input. The trust tiers handle the case the others miss: a claim that arrived clean, looks plausible and is false.

Where this fits with layered memory

The tiers are fancier than they sound, and they slot into a standard memory taxonomy. Semantic memory holds facts and knowledge, episodic memory holds events and interactions, and procedural memory encodes learned skills (the memory types overview). The trust split applies mostly to semantic memory, which is where a wrong fact does the most damage. The same governance reasoning supports designing memory deliberately rather than letting it accumulate, a point made at length in the agent memory design guide.

The hierarchical-memory view helps explain why the split is cheap. MemGPT's core idea is to manage memory tiers with explicit paging between a small in-context working set and a larger external store (the MemGPT paper, summary). Adding a permission boundary per tier costs almost nothing in that architecture. The agent proposes a write, the runtime routes it to draft, and the answer path pages only from read-only. The same paper reports a jump in deep-memory retrieval accuracy for GPT-4 from 32.1% to 92.5% with MemGPT, though the authors note the baseline received a lossy summary while MemGPT could search full history, so the comparison is not a clean isolation of the architecture (alphaXiv overview).

One architectural note on the answering path. Whether you compose answers with a hosted model, a local model or a mix, the permission boundary should sit in the data layer, not in the prompt. Point the answer path at a read-only table and the constraint holds regardless of which model is behind the call. Quanta Inference (https://quantaintelligence.ai/inference) is one way to reach text, image and embedding models through a single OpenAI-compatible key if you want that routing to stay simple.

Rollout checklist

Start narrow and measure, because the failure you are fixing is quiet.

  • Pick ten high-value fields for the promotable list. Leave payment destinations, health data and access levels out.
  • Add source_id, source_kind, observed_at and parent_source_id to every write. Lineage is not optional.
  • Enforce permissions in the database, not in application code. Test with a connection using the answer role.
  • Route every untrusted write to draft. Keep authoritative writes out of draft entirely.
  • Run the promotion worker on a schedule and log each decision with its reason.
  • Build the review queue before you need it. Count the burden per week.
  • Add a tombstone on demotion so a rejected value cannot return.
  • Measure draft table growth. A draft table that never drains is a signal, not a bug.

For teams that want the schema, worker and review console built alongside their existing stack, Quanta (https://quantaintelligence.ai/) takes a described system and builds it with you, including the permission model.

FAQ

FAQ

Does the answer path ever need to see draft rows?

No, and that is the design. Draft rows exist for the promotion worker and for audit. The one exception worth considering is surfacing a draft to the user as "noted, not yet used," which keeps the agent honest without granting the claim any authority in an answer.

How is a second independent source actually checked?

By comparing source_id values and walking parent_source_id lineage back to raw evidence. Two rows count as independent only if neither descends from the other. A summary, restatement or tool echo of an original does not create a new source (Zep).

What happens to a draft that is never confirmed?

It stays in draft, and a periodic job decides whether to keep or drop it. Dropping unconfirmed drafts after a set window is reasonable because nothing downstream ever trusted them. The draft table is where TTL is genuinely safe to apply.

How does this differ from quarantining bad chunks at ingest?

Chunk quarantine judges a passage when it arrives and only sees what is visible in the text. Trust tiers judge a claim against other evidence over time and keep the answer path clean even when the claim arrived looking harmless. They catch different failures and should run together.

Is promotion ever wrong in the other direction, rejecting true facts?

Yes, and the published numbers say so. In GovMem's project-internal subset, dependency-aware promotion held false promotion to 0.032 overall but direct recall fell to 0.448, meaning many useful facts stayed stuck behind review (the GovMem paper). Size the review queue accordingly and keep a manual promote path for trusted operators.

Sources

Sources

  1. Memory Poisoning Attacks on Large Language Model Agents
  2. Memory poisoning in AI agents: exploits that wait
  3. Defending Agent Memory Against Poisoning | Zep Blog
  4. How Agents Remember, and Who Gets to Write It
  5. When Not to Write Memory: Governing False Promotion ...
  6. Agent Memory Atlas
  7. Memory eviction and forgetting in AI agents
  8. agent-memory/docs/how-to/configure-ttl.md at main
  9. Semantic vs Episodic vs Procedural Memory in AI Agents - Medium
  10. AI Agent Memory Design Guide - Working, Long-Term, and ...
  11. 2310.08560 MemGPT: Towards LLMs as Operating Systems - arXiv
  12. MemGPT: Hierarchical Memory Management - Emergent Mind
  13. MemGPT: Towards LLMs as Operating Systems - alphaXiv
About the author

About the author

Antonio Evans
Antonio Evans

Antonio Evans is the founder of Quanta Intelligence and a fractional CTO who builds HIPAA-ready AI systems for health and life-science companies. Before that he was technical co-founder of FriendsEAT for twenty years, CEO of Socialdraft, and a data engineer at IBM. Antonio lives with his dog Santino in Lisbon, Portugal.

All articles