Human conversation turns over in about 200 milliseconds. Everything a voice agent does has to fit in that gap.
There is published research on how fast people answer each other, and it sets a budget most systems cannot meet.
Human conversational turn-taking is remarkably fast and remarkably consistent. Stivers and colleagues examined ten unrelated languages and found response offsets clustering in a single peak within 200 milliseconds of a question ending, with an overall mode of 0 milliseconds and a cross-linguistic median around +100. That is not a preference people have. It is the timing they are calibrated to, and a pause noticeably longer than it reads as hesitation, confusion, or a dropped call.
What has to happen inside that gap?
Four things, in sequence, each taking real time: detecting the caller stopped, transcribing what they said, deciding what to say back, and generating speech. They are serial (each waits on the one before) which is why total latency is the sum rather than the maximum.
The endpointing tradeoff is the interesting one, because it cannot be optimised away; it can only be moved. Waiting longer before responding makes the system worse at feeling immediate and better at not interrupting people who pause mid-sentence. Waiting less does the reverse. Any system claiming to have solved both has picked one and not told you which.
Integrations are where latency quietly appears in production. A line that answers instantly in a demo and pauses for two seconds before offering an appointment is not slow at conversation; it is waiting on a booking system. That delay belongs to the integration, and the fix is usually to speak first and look up while talking.
| Stage | What it does | Why it costs time |
|---|---|---|
| Endpointing | Decides the caller has stopped speaking | Must wait to be sure a pause is not a thinking pause |
| Recognition | Turns audio into text | Can stream, but final text needs the end of the utterance |
| Reasoning | Decides the response, checks a calendar, looks up a record | Any external lookup adds its own round trip |
| Speech synthesis | Turns text into audio | Can stream from the first words, which is where most latency is won |
Endpointing is the stage most often blamed on "the AI being slow" when it is actually a deliberate wait to avoid cutting the caller off.
Is it total latency callers notice, or something else?
Mostly it is silence: unexplained, unfilled dead air. A gap that a human would have filled with an acknowledgement reads as a fault, while the same delay covered by "let me check that for you" reads as someone doing their job.
This is why the engineering answer and the experience answer diverge. Cutting 300 milliseconds off a pipeline is hard and expensive. Speaking within 200 milliseconds and then continuing to talk while the lookup runs is a design change, and it removes the same complaint.
The other thing callers notice is inconsistency. A line that answers instantly nine times and then pauses for three seconds is worse than one that always takes a beat, because the pause is unexpected and reads as a failure rather than a rhythm.
Why is this harder on a phone than in a browser demo?
Because the phone network adds its own delay before the system does anything. Call setup, codec processing and carrier routing all sit underneath, and none of it is under the agent's control; the software budget is what is left after the network takes its share.
It is also why comparing a browser-based voice demo to a real phone call is misleading in the vendor's favour. The demo has no carrier in the path. The phone call does, and it is the phone call that has to work.
Across ten unrelated languages, people answer each other with a response peak inside 200 milliseconds, so a voice agent is not being judged against other software, it is being judged against the timing every caller has been calibrated to since infancy.
Latency is the thing you cannot judge from a demo video. Call a line and listen to the gaps.
Start free pilotWhat this page does not claim
The research cited is about human conversation, not about any particular voice system's performance.
- No latency figure for our lines. We do not currently publish measured end-to-end latency, so this page does not quote one. A number without the measurement conditions would not mean anything anyway.
- No threshold for when AI 'feels wrong'. The 200ms figure describes how fast humans answer each other. Extrapolating it into a pass/fail line for machines is an inference, and this page does not present it as a finding.
Questions this raises
- How fast do people actually respond in conversation?
- Stivers et al. found a unimodal peak of responses within 200 milliseconds of a question ending, across ten languages, with an overall mode of 0ms and a cross-linguistic median near +100ms.
- Why does a voice agent pause before answering?
- Usually endpointing, the deliberate wait to confirm the caller has finished rather than paused. Reducing it makes the system faster and more likely to interrupt people mid-sentence.
- Why is the pause longer when it checks a calendar?
- Because an external lookup adds a round trip the conversation has to wait for. The usual fix is conversational rather than technical: acknowledge first, look up while speaking.
- Is a browser demo representative of phone latency?
- No. A phone call carries carrier routing and codec processing that a browser demo does not. The software budget is whatever remains after the network takes its share.
- What matters more, average latency or consistency?
- Consistency, in our view. An unexpected three-second gap after a run of instant replies reads as a fault, where a steady short beat reads as a rhythm.
Sources
- Universals and cultural variation in turn-taking in conversationStivers et al., PNAS 106 (26)2009, consulted 2026-08-09
Want an AI receptionist built around these rules rather than despite them?
We build a working receptionist on your calls, your booking rules and your escalation rules, and you call it yourself before committing to anything. About 14 days, no call limit during the pilot.
Start free pilot