Memory for AI Agents: What to Store and What to Forget

Most writing on the subject uses four categories, taken from the CoALA framework published by Princeton researchers in 2023 (Atlan, Types of AI agent memory). Redis describes the split between short-term and long-term memory with a hardware analogy: short-term memory is like RAM and lasts for one conversation thread, while long-term memory is like a hard drive and persists across sessions (Redis, Short-term and long-term memory for AI agents).

Not every agent needs all four. Redis gives a useful matching rule: a support agent needs episodic memory of past tickets, a product recommendation agent needs semantic memory of products and preferences, and a coding assistant benefits from procedural memory of debugging approaches that worked (Redis). Start with the one your use case clearly needs and add others when you can name the failure they fix.

Memory is also different from retrieval-augmented generation (RAG). Mem0 puts the distinction simply: "RAG helps the agent answer better. Memory helps the agent behave smarter." RAG pulls facts from a shared document corpus. Memory is an evolving record built from one user's or one task's past interactions, and most production agents use both (Mem0, AI agent memory guide).

The best filter is a single question: will this change a future decision? If the answer is no, do not store it. Redis lists deciding "what information deserves persistence versus what can be discarded" as one of the decisions teams most often underestimate (Redis).

Good candidates:

• Stable preferences the user stated. Language, format, tone, tools they use, things they asked you never to do.
• Facts about the account or project that the agent would otherwise have to ask for again.
• Decisions and their reasons. "Chose plan B because the client's budget is fixed" saves a repeat debate.
• Lessons from failures. "The export API times out on files over a certain size, so split them first" is procedural memory that prevents a repeat.
• Open commitments. Anything the agent promised to follow up on.

Poor candidates:

• Raw transcripts. Keep them in logs for audit, not in the memory the agent reads.
• The agent's own guesses. An inference such as "the user is probably a manager" becomes a fact the moment it is stored.
• Sensitive data the task does not need. Health details, financial identifiers and third-party personal data carry risk every day they sit in a store.
• Anything already available from a system of record. If the CRM holds the customer's plan, look it up instead of copying it into memory where it will go stale.

Mem0 flags why the sensitive-data rule matters most for semantic memory. Semantic memory is designed to be durable, so a fact like a user's employer or health condition "persists indefinitely unless something explicitly updates or deletes it" (Mem0, Semantic memory for AI agents). Store less, and you have less to protect, correct and delete.

The naive approach appends every candidate fact to a list. Within weeks the list is full of duplicates and contradictions. A better pattern has two steps: extract, then consolidate.

AWS describes this pipeline for its AgentCore memory service, and the design applies to any stack. An extraction step runs asynchronously after the conversation, using a model to pull out facts, preferences and session summaries in a defined schema. Then, for each extracted memory, a consolidation step retrieves the most similar existing memories and asks a model to choose one of three actions (AWS, AgentCore long-term memory deep dive):

Conflicts resolve in favor of the newer information, but the old record is kept. AWS's example: an existing memory says the customer budget is 500, a later conversation says it increased to 750, and the result is an active memory of 750 with the previous one marked inactive. Outdated memories are marked INVALID rather than deleted, which keeps an audit trail (AWS).

Mem0 describes the tension this design resolves. Hard deletes on every contradiction lose the history, while soft writes on every utterance create duplicates, and many teams swing between the two before settling on a hybrid (Mem0, Semantic memory).

Three rules make the write path safer:

1. Write asynchronously. Extraction should not slow down the user's reply.
2. Store provenance. Record when each memory was written, from which session and whether the user stated it or the agent inferred it.
3. Keep episodic and semantic shapes separate. Mem0 warns that storing "the user prefers Postgres" and "the user mentioned Postgres on March 4" the same way turns durable preferences into noisy history.

Retrieval decides whether stored memory helps or hurts. Pull too much and you are back to a bloated prompt. Pull the wrong thing and the agent acts on irrelevant or stale facts.

Redis recommends hybrid retrieval: structured lookups first, using exact matches on user ID, preference keys and timestamps, then vector search as a second pass for semantic relevance. It also suggests starting with plain vector search and adding complexity only when retrieval quality becomes the bottleneck (Redis).

Scope comes before relevance. Every read must filter by the owner of the memory: the user, the team or the tenant. Mem0 calls cross-user fact contamination "one of the rare memory bugs that is both subtle and a security incident at the same time," because without strict scope keys a fact stored for one tenant can appear in another tenant's prompt (Mem0, Semantic memory). Enforce the scope in the query layer, not in the prompt.

def recall(user_id: str, query: str, k: int = 5):
    # Scope first: only this user's active memories.
    candidates = store.filter(owner=user_id, status="active")
    # Then rank by similarity, with a boost for recent memories.
    return candidates.search(query, top_k=k, recency_weight=0.2)

Two other choices matter. First, decide whether memory is pushed or pulled. Pushing means your code inserts relevant memories into every prompt. Pulling means the agent has a tool to search memory when it decides it needs to, which is the approach MemGPT takes (MemGPT). Pushing is simpler and more predictable. Pulling scales better when memory is large. Second, cap the amount of memory per call and label it clearly in the prompt, so the model can tell remembered context from the user's current request.

Forgetting is not a cleanup task for later. Without it, Redis notes, memory systems grow without bound and retrieval quality degrades as irrelevant memories pollute results. Its two practical approaches are timestamps with recency weighting during retrieval, and automatic expiry of old data, and the two combine well (Redis).

Different memory types deserve different policies:

Stale facts are the most common failure. Mem0's example: the user changes jobs or switches tools, nothing retires the old fact, and the agent "suggests an outdated framework with full confidence" (Mem0, Semantic memory). Defend against it in three ways. Let the consolidation step mark old facts inactive when later ones contradict them. Re-confirm high-impact facts before acting on them, for example "You mentioned a budget of 750 last month. Is that still right?" And give users a way to see and edit what the agent remembers about them.

Deletion has to be real. Mem0 points out that a memory layer that can only add facts "accumulates PII indefinitely with no mechanism to remove it on request," while explicit update and delete operations give users a real path to correct or remove what is stored (Mem0, Semantic memory). When a user asks to be forgotten, delete from the memory store, the vector index and any summaries derived from their data.

Instruction files drift too. Mem0 notes that hand-edited instruction files read on every turn tend to become bloated and contradictory after a few months. Review them on a schedule the same way you review stored memories.

If you are adding memory to an agent for the first time, this design covers most needs without much machinery:

1. Working memory lives in the conversation state for one session, with old tool output summarized or dropped as the session grows.
2. A small semantic store per user holds stated preferences and key facts, written by an asynchronous extract and consolidate job after each session.
3. An episodic log keeps short summaries of past sessions with dates, retrieved by recency and similarity and expired after a set period.
4. Every read filters by owner and active status before ranking.
5. Every memory carries a timestamp, source session and a stated or inferred flag.
6. Users can view, edit and delete what is stored about them.

Then measure. Track how often retrieved memories are used in the answer, how often users correct the agent about something it remembered and how large each user's store grows. Those numbers tell you when to add procedural memory, change decay settings or move to agent-driven retrieval.

If your agents work from shared playbooks, keep those in a separate, versioned store from per-user memory. Shared skills and playbooks, such as the ones Quanta Intelligence publishes, belong in that shared store, never in one user's memory.