Giving an email agent memory of the people it talks to

Ask people what makes automated email infuriating and they will describe a memory failure. It asked for my order number again. It offered me a plan I already have. It introduced itself for the fourth time. None of those are language problems, and a better model does not fix any of them.

27 July 2026 · 5 min read

Three kinds of memory, and where each belongs#

Thread memory is what was said in this conversation. It comes from get_thread and costs nothing to maintain, because the mail system already holds it. Use it always, and never store your own copy: two records of one conversation diverge the first time somebody replies from their phone.

Person memory is what you know about this correspondent across conversations. It lives on the contact, is shared across the workspace, and is the memory most agents are missing.

Workspace memory is what is true regardless of who is writing: policies, limits, product facts, escalation rules. It lives in notes, and it is how several agents stay consistent with each other.

Getting these in the right place matters more than any of them individually. Person facts in the system prompt go stale and cost tokens on every call. Policy in a contact record is invisible to every other conversation.

The loop that produces memory#

Memory is not a feature you switch on, it is two calls you make every time.

lookup_contact        { email: "anna@customer.com" }     // before writing
remember_about_contact { contact_id: "con_...", fact: "Prefers delivery to the office, not home." }

The first call is the one that prevents the irritating questions. The second is the one that makes the next conversation better, and it is the one people forget, because nothing breaks when you skip it. Nothing breaks for weeks, and then your agent is still asking a customer of two years to confirm which product they use.

Make writing a fact part of finishing a thread, alongside replying and archiving, as in the system prompt.

What is worth remembering#

The temptation is to record everything, which produces a contact record nobody can use and a context window full of noise. Record what would change a future reply.

The test: would this still be true and useful in three months. If not, the thread already has it.

Do not duplicate your own database#

An agent should read the current state of an order from your order system, not from a fact recorded in June. Contact memory is for what your systems do not hold: preferences, relationship context, how someone likes to be dealt with. Facts that mirror a database row become wrong silently, which is worse than not having them.

A good division: your systems know the state, the contact knows the person. What belongs in the mailbox itself, and how to keep it usable at volume, is in keeping an agent inbox usable.

Sharing memory across agents#

Contacts are a workspace resource, so a fact recorded by the support agent is available to the billing agent when you grant it. That is most of the argument for running agents in one workspace rather than four accounts, and the sharing boundaries are worked through in running several email agents in one workspace.

Two cautions. Facts recorded by one agent are read by another with no context about why, so write them as standalone statements rather than notes to self. And be deliberate about what belongs in a shared record at all: something a customer told support in confidence is not automatically something a sales agent should open with.

Keeping the address book clean#

Contact books rot. The same person writes from a personal address and a work one, a company gets acquired, someone's name changes. find_duplicate_contacts surfaces likely pairs and merge_contacts combines them, keeping the facts from both, which is the maintenance job worth running periodically rather than never.

Import matters too. import_vcards brings in an existing book so an agent starts with what your team already knows rather than from zero, and export is available the same way, which is the answer to the reasonable question of whether this data is locked in.

The privacy side, briefly#

Recorded facts about identifiable people are personal data, and the obligations that come with that are not suspended because an agent wrote them. Keep facts relevant to the work, keep them accurate, delete what stops being useful, and be able to answer a deletion request. Notes about a person are covered by the same rules as anything else in your CRM, and an agent that writes a snide observation about a customer has created a record you will one day have to hand over.

Write facts you would be comfortable showing the person they describe. That single rule keeps you clear of most of the trouble, and the trouble it avoids is expensive.

Questions#

How does an AI email agent remember previous conversations?

Through three layers: the thread itself for what was said, contact records for what is known about the person across conversations, and notes for workspace knowledge. lookup_contact before writing and remember_about_contact when finishing.

What should the agent record?

Durable things that would change a future reply: preferences, role, constraints, commitments made. Not summaries of individual messages, which the thread already holds.

Should facts live in the system prompt instead?

No. Prompts are for behaviour, contacts are for people. Facts in a prompt go stale, cost tokens on every call, and cannot be shared between agents.

Can several agents share contact memory?

Yes. Contacts are a workspace resource, so a fact recorded by one agent is available to others you grant access. Write facts that stand alone, since the reading agent has none of the original context.

How do I stop the contact book filling with duplicates?

Run find_duplicate_contacts periodically and merge_contacts on the pairs, which keeps the facts from both records.

Is recorded contact data subject to privacy rules?

Yes. Facts about identifiable people are personal data regardless of who wrote them, so keep them relevant and accurate, delete what stops being useful, and be able to honour a deletion request.