Setting response time targets you can actually keep
The first thing an email agent changes is response time, and the first mistake teams make is promising the new number publicly. The agent answers in seconds for the questions it handles, and the moment something escalates you are back to a human's timeline, which is now sitting under a published commitment nobody agreed to.
4 min read
Two clocks, not one
Every inbox with an agent on it has two: the agent's, measured in seconds, and the human's, measured in hours. A single published target has to be honest about the slower one, because the customer does not know which path their message took.
The way out is per-category targets rather than one number.
| Category | Answered by | Realistic target |
|---|---|---|
| Documented question | Agent | Under a minute |
| Order or account status | Agent, from your systems | Under a minute |
| Anything needing a decision | Human, after triage | Same working day |
| Money above the limit | Human | Same working day |
| Legal, security, complaints | Named human | Under four hours, and say so |
The valuable commitment is not the fast one. It is the promise that *everything* gets an acknowledgement quickly, even when the real answer takes a day, which is a thing an agent can guarantee and a team cannot.
Acknowledge fast, resolve honestly
Split the target in two. Time to acknowledgement is under a minute, always, and that acknowledgement is specific: what will happen, who is picking it up, and when to expect them. Time to resolution is per category and honest.
reply_all {
message_id: "msg_...",
body: "Thanks, I have passed this to Maria on our billing team since it involves a credit.\n\nShe will reply today. If you need it sooner, say so here and I will flag it."
}That message costs nothing, is true, and removes the silence that makes people write again. It also stops the second most expensive thing in a support inbox, which is duplicate threads from the same person.
Measure the tail
Averages hide the problem. A median of thirty seconds means nothing when the 95th percentile is nine hours, because the customer who waited nine hours is the one who writes the review.
Watch the 95th percentile per category, split by whether the agent or a human answered. Two failure shapes appear there and they need different fixes: a long tail on agent-handled threads means your routing or webhook path is broken, and a long tail on escalations means the human side is under-staffed for the volume the agent is sending it. The metric set is in evaluating an email agent.
Design the fallback before you publish anything
Three things will happen, and a published target has to survive all of them.
The agent stops working. Silence looks exactly like a quiet inbox, so a synthetic check that sends a message and asserts a reply is what catches it, per what to monitor in production.
Escalations pile up. Someone is away, a queue is not drained. Anything older than your promised time should page a person rather than sitting, per designing approval that a human will still read.
Volume spikes. A burst degrades into a queue rather than a wall, provided the concurrency work is done, per what happens when forty messages arrive at once.
Out of hours, honestly
An agent answers at 3am, which is genuinely useful and creates an expectation that the escalation path cannot meet. Say the boundary in the message rather than leaving it implied: an acknowledgement at 3am should state when a person is next available, not imply that one is reading.
The same applies to weekends and holidays. An agent that answers cheerfully on 24 December while nobody can escalate for four days is setting up a bad conversation. Either widen the agent's remit for those windows deliberately, or say plainly what happens next and when.
What to publish
Less than you can do. Publish the acknowledgement guarantee and a resolution target you would hit on your worst week, not your best. Then let the actual performance beat it, which is the only version of this that builds trust rather than spending it.
Internally, track the real numbers per category and per path. Externally, one honest sentence is enough: everything gets an answer within the hour, and anything needing a colleague gets one the same working day.
Questions
- What response time can an AI email agent hit?
- Under a minute for anything it handles from documentation or your systems. Escalations run on your team's clock, which is why a single published number is usually a promise you cannot keep.
- Should I publish the agent's speed?
- Publish an acknowledgement guarantee rather than a resolution one. Everything gets a specific reply quickly; the real answer follows on a per-category timeline you would hit on a bad week.
- Which metric should I watch?
- The 95th percentile per category, split by agent-handled and human-handled. The median flatters, and the tail is what customers write about.
- What does a long tail on agent-handled threads mean?
- Usually a broken path rather than a slow model: a webhook not arriving, a queue not draining, or an expired key. A synthetic check catches it before a customer does.
- How should out of hours work?
- State when a person is next available rather than implying someone is reading. An agent answering at 3am creates an expectation the escalation path has to meet.
- What about the weekend?
- Decide deliberately: either widen what the agent handles for that window, or say plainly what happens next and when. Cheerful answers with a four-day escalation gap are worse than silence.
Give your agent an address it can answer from.
Create an inbox