---
title: Giving an email agent memory of the people it talks to
metaTitle: Agent memory for email: contacts, facts, and notes
description: The difference between an agent that feels competent and one that feels robotic is memory. How to use contacts, recorded facts, and notes so the second conversation starts where the first ended.
date: 2026-07-27
author: MCPmailer
tags: Guides
---

Ask people what makes automated email infuriating and they will describe a memory failure. It asked for my order number again. It offered me a plan I already have. It introduced itself for the fourth time. None of those are language problems, and a better model does not fix any of them.

![A contact record accumulating facts across several conversations](/blog/agent-memory-contacts-email/hero.webp "The address is the key. Everything learned about a person hangs off it.")

## Three kinds of memory, and where each belongs

**Thread memory** is what was said in this conversation. It comes from `get_thread` and costs nothing to maintain, because the mail system already holds it. Use it always, and never store your own copy: two records of one conversation diverge the first time somebody replies from their phone.

**Person memory** is what you know about this correspondent across conversations. It lives on the contact, is shared across the workspace, and is the memory most agents are missing.

**Workspace memory** is what is true regardless of who is writing: policies, limits, product facts, escalation rules. It lives in notes, and it is how several agents stay consistent with each other.

Getting these in the right place matters more than any of them individually. Person facts in the system prompt go stale and cost tokens on every call. Policy in a contact record is invisible to every other conversation.

## The loop that produces memory

Memory is not a feature you switch on, it is two calls you make every time.

```text
lookup_contact        { email: "anna@customer.com" }     // before writing
remember_about_contact { contact_id: "con_...", fact: "Prefers delivery to the office, not home." }
```

The first call is the one that prevents the irritating questions. The second is the one that makes the next conversation better, and it is the one people forget, because nothing breaks when you skip it. Nothing breaks for weeks, and then your agent is still asking a customer of two years to confirm which product they use.

Make writing a fact part of finishing a thread, alongside replying and archiving, as in the [system prompt](/blog/email-agent-system-prompt).

## What is worth remembering

The temptation is to record everything, which produces a contact record nobody can use and a context window full of noise. Record what would change a future reply.

| Worth a fact | Not worth a fact |
| --- | --- |
| A durable preference: contact time, channel, address | The content of one message |
| Their role, and who else to copy | Anything already in the thread |
| A commitment you made to them | Sentiment on one day |
| A constraint: contract terms, plan, region | A summary of the conversation |
| A correction they had to make twice | Anything you can derive from your own systems |

The test: would this still be true and useful in three months. If not, the thread already has it.

## Do not duplicate your own database

An agent should read the current state of an order from your order system, not from a fact recorded in June. Contact memory is for what your systems do not hold: preferences, relationship context, how someone likes to be dealt with. Facts that mirror a database row become wrong silently, which is worse than not having them.

A good division: **your systems know the state, the contact knows the person.** What belongs in the mailbox itself, and how to keep it usable at volume, is in [keeping an agent inbox usable](/blog/agent-inbox-search-and-archive).

## Sharing memory across agents

Contacts are a workspace resource, so a fact recorded by the [support agent](/blog/ai-agent-customer-support-email) is available to the [billing agent](/blog/automated-invoice-chasing-ai-agent) when you grant it. That is most of the argument for running agents in one workspace rather than four accounts, and the sharing boundaries are worked through in [running several email agents in one workspace](/blog/multi-agent-email-workspace).

Two cautions. Facts recorded by one agent are read by another with no context about why, so write them as standalone statements rather than notes to self. And be deliberate about what belongs in a shared record at all: something a customer told support in confidence is not automatically something a sales agent should open with.

![A fact recorded by one agent being used by another](/blog/agent-memory-contacts-email/sharing.webp "Written by one agent, read by another, with no shared context. Write facts that stand alone.")

## Keeping the address book clean

Contact books rot. The same person writes from a personal address and a work one, a company gets acquired, someone's name changes. `find_duplicate_contacts` surfaces likely pairs and `merge_contacts` combines them, keeping the facts from both, which is the maintenance job worth running periodically rather than never.

Import matters too. `import_vcards` brings in an existing book so an agent starts with what your team already knows rather than from zero, and export is available the same way, which is the answer to the reasonable question of whether this data is locked in.

## The privacy side, briefly

Recorded facts about identifiable people are personal data, and the obligations that come with that are not suspended because an agent wrote them. Keep facts relevant to the work, keep them accurate, delete what stops being useful, and be able to answer a deletion request. Notes about a person are covered by the same rules as anything else in your CRM, and an agent that writes a snide observation about a customer has created a record you will one day have to hand over.

Write facts you would be comfortable showing the person they describe. That single rule keeps you clear of most of the trouble, and the trouble it avoids is expensive.

## Questions

### How does an AI email agent remember previous conversations?

Through three layers: the thread itself for what was said, contact records for what is known about the person across conversations, and notes for workspace knowledge. `lookup_contact` before writing and `remember_about_contact` when finishing.

### What should the agent record?

Durable things that would change a future reply: preferences, role, constraints, commitments made. Not summaries of individual messages, which the thread already holds.

### Should facts live in the system prompt instead?

No. Prompts are for behaviour, contacts are for people. Facts in a prompt go stale, cost tokens on every call, and cannot be shared between agents.

### Can several agents share contact memory?

Yes. Contacts are a workspace resource, so a fact recorded by one agent is available to others you grant access. Write facts that stand alone, since the reading agent has none of the original context.

### How do I stop the contact book filling with duplicates?

Run `find_duplicate_contacts` periodically and `merge_contacts` on the pairs, which keeps the facts from both records.

### Is recorded contact data subject to privacy rules?

Yes. Facts about identifiable people are personal data regardless of who wrote them, so keep them relevant and accurate, delete what stops being useful, and be able to honour a deletion request.

## Related

- [The system prompt for an email agent](/blog/email-agent-system-prompt)
- [Running several email agents in one workspace](/blog/multi-agent-email-workspace)
- [Contacts, notes, and vault](/docs/contacts-notes-vault)
- [How to give an AI agent its own email address](/blog/email-for-ai-agents)