What an AI Agent Should Remember

“We moved the launch to the 22nd. Why does the assistant still say the 8th?”

Imagine a team that discussed the first date in chat, changed it in a planning meeting and approved the new date in its release tracker. An assistant that remembers the chat beautifully can still give the wrong answer.

That is what makes memory interesting to me. Remembering more is not automatically useful. The system needs to know what kind of information it has, where it came from and when to check again.

A bigger conversation is only one part of memory

The context window is the material available to the model for a particular response: instructions, messages, documents and tool results. An application can also save information outside that window and retrieve it later. Saving a note in a database does not, by itself, train the model.

Anthropic’s context engineering guide describes selective retrieval and structured notes as ways to manage what reaches the model. The part I find useful is the emphasis on choosing the material for the current task, rather than treating the whole history as equally relevant.

For the launch assistant, I would separate three things before choosing a memory library.

Working state is where we are in the task. Which release are we discussing? What has been checked? What is still unresolved? It should help the next step without becoming a permanent claim about the user or company.

Reusable memory holds things that are useful across tasks, such as an explicitly approved preference for short release summaries. A preference needs a scope. One person’s writing preference should not silently become the team’s policy.

The source system holds facts that change: release dates, delivery status, account permissions or whether a document has been approved. I would store a link or identifier and fetch the current record when the answer depends on it.

Those distinctions can live in a relational database, files or an existing application. Semantic search may help find relevant material by meaning. It does not decide which version is authoritative.

Keep the receipt with the memory

A saved statement such as “Launch is on the 8th” loses the details needed to judge it. Was that an early suggestion, an approved decision or the assistant’s own inference?

For information worth keeping, I would want a small record:

Not every note needs every field. A temporary checklist can be much simpler. But if the system cannot say where a persistent belief came from, correcting it will be harder than it needs to be.

For the launch example, I would define the approved release tracker as the authority for the current date. The meeting notes explain why it changed. The old chat remains historical context. “Pick the newest text” would be a poor substitute for that rule because a new casual message might still be wrong.

Write a handover another session can use

An agent that works across several sessions needs a useful way to resume. I would save the objective, completed work, unresolved questions and links to evidence. “I made good progress” is not a handover.

Anthropic’s work on long-running agents uses progress records and other durable artefacts to help later sessions continue. That is an engineering account of one approach, rather than proof that every agent needs the same setup.

For a planning assistant, a helpful note might say: “Checked the release tracker. The approved date is 22 November. The customer announcement is still awaiting review.” It should link to the record and avoid turning “awaiting review” into “approved” during the next summary.

Summaries deserve checking too. Repeatedly shortening a conversation can remove the disagreement, condition or uncertainty that made a statement accurate. Keep a route back to the original evidence for decisions that matter.

Test change, forgetting and uncertainty

The LongMemEval benchmark tests several abilities, including reasoning across sessions, handling knowledge updates and declining to answer when information is missing. That is a useful way to think about memory evaluation. I would borrow the question types without treating an older benchmark score as a verdict on today’s models.

Before adding more storage, I would run five small conversations through the application:

1. Change an agreed fact. Does the next answer use the approved update?
2. Put a preference in one person’s account. Does another person ever see it?
3. Ask about something never supplied. Does the assistant admit the gap?
4. Remove a stored preference. Does it stop influencing later answers, including through retrieved summaries?
5. Resume an unfinished task. Can the assistant identify what remains without declaring it complete?

For deletion, define what the user action covers: the original record, search index, derived summaries and any retained history. A “forget” button should have behaviour the team can explain and test.

You can start this exercise on paper, with no paid service. Choose one assistant task and write down what it may keep, what it must fetch again and what it must never carry between users. Then try to break those rules with a changed decision.

That gives me a clearer starting point than choosing a memory database. I want the assistant to bring the right information back at the right moment, and let it go when it no longer belongs.

References

• Anthropic: Effective context engineering for AI agents
• Anthropic: Effective harnesses for long-running agents
• Wu et al.: LongMemEval, ICLR 2025