Roman Poliashenko
We build voice agents that run all day on real calls, so let me be blunt: when a client says "it sounds like a robot," 95% of the time the problem isn't the voice. In 2026 the timbre is already nearly indistinguishable from human — that's table stakes, available to everyone. What makes an agent a robot is timing: the exact moment it starts speaking, how long it waits before deciding you're done, and what it does when you interrupt it. It's the dull, invisible mechanics of conversation — and it's exactly what gives the machine away.
And there's a counterintuitive trap. It seems logical: "the agent replies with a delay — make it faster." But "faster" applied blindly often means the agent starts cutting off a live human mid-sentence. Let's look at where the real bottleneck is — and what actually works.
A human doesn't react to the pause — it predicts it
Start with how a live conversation is wired. In the landmark Stivers et al. study (PNAS, 2009), across 10 languages the average gap between speakers' turns is about 200 ms, with language-specific means falling within ~250 ms of the overall figure. People hand the floor to each other with almost no pause — from Japanese (about 7 ms on average) to Danish (about 470 ms).
Now the crucial part. Producing even a short one-word answer takes a human at least ~600 ms to plan and articulate. Yet the reply comes back in 200 ms. That's physically possible under only one condition: the human doesn't wait for you to fall silent — it predicts when and how you'll finish your turn, and starts preparing the answer mid-sentence. Live conversation runs on prediction, not on reacting to silence.
That's where "roboticness" is born. A typical voice agent does the exact opposite: it waits for silence, runs a mute timer, and only then starts thinking. It's reactive by construction — which means it's structurally always half a beat behind a human. The best TTS in the world won't hide that.
Why "just make it faster" is not the fix
The industry has built clear benchmarks around this. A response within ~300 ms feels natural — exactly the human gap between turns. Beyond ~500 ms the illusion of a live dialogue falls apart: people start repeating themselves, re-asking, or going quiet. And after roughly 1.2 s of silence, the other party simply talks over the agent or hangs up.
You'd think — just drive the latency down and you're done. But here's where it actually hides in the classic "recognition → model → synthesis" pipeline: STT takes about 100–500 ms, the LLM (time-to-first-token) anywhere from 200 ms to 2 s, TTS 100–400 ms, plus network at 20–100 ms each way. The key point: the largest and most noticeable delay sits not in STT and not in TTS, but in two places — the LLM's time-to-first-token and, above all, the decision that you're done talking. That's exactly why end-to-end speech models (for instance, GPT-4o's native speech-to-speech, with a median voice-to-voice latency of about 320 ms) are pressing on the market: they remove the intermediate seams and close in on the human 200–300 ms. But even for them, the central question remains — when to start speaking.
The endpointing trap
Here's the real bottleneck. To reply, the agent has to decide that you've finished speaking. The most common method is VAD (voice activity detection): the agent listens for silence, and when the quiet lasts longer than a threshold, it treats your turn as over. The problem is that VAD doesn't understand meaning — it only hears the presence or absence of sound. And that creates a bind that "faster" can't solve:
- Set the silence threshold short — the agent becomes "fast," but it starts cutting you off every time you pause to think mid-sentence: "my number is... (pause) ...380...". To VAD, that pause = end of turn. A false interruption.
- Set the threshold long — the agent stops interrupting, but now there's a dead pause hanging after every sentence you say. It feels slow and dull.
No single-slider setting wins on both at once. The way out isn't to tune the threshold — it's to give the agent what a human has: an understanding of meaning. Semantic end-of-turn detection (a model reads the partial transcript and judges whether the phrase is complete in meaning, not just whether silence has fallen) lets the agent wait an extra fraction of a second when you clearly aren't done ("my number is..."), and answer instantly when the thought is complete. That's the machine analogue of human prediction — and it, not a "faster voice," is what removes the roboticness.
Barge-in: a half-second test of humanity
There's one more thing that instantly gives a robot away — what the agent does when you interrupt it mid-sentence. A live human goes quiet almost immediately once you start speaking. A poorly configured agent either keeps monologuing over you (the worst feeling — like talking to a wall), or, conversely, gets silenced by any "uh-huh" or cough, cutting off its own useful answer. Correct barge-in handling — so the agent goes quiet exactly when you genuinely take the floor and ignores background acknowledgments — is a separate engineering task that has nothing to do with "what its voice sounds like."
Why this isn't just about feel — it's about money
On a phone call, timing is conversion. A call has no face and no on-screen text: if the agent stumbles, interrupts, or leaves dead pauses, the person has no other signal to go on — and they hang up. Every extra half-second of delay and every false interruption isn't "aesthetics" — it's a share of customers who never reach the booking, the order, or the payment. So timing isn't cosmetics layered onto a finished agent; it directly decides whether the voice channel pays off at all.
How we see it (an integrator, not a box)
When we launch a voice agent, "which voice" is about the easiest decision in the project. The real work is in timing engineering: matching the end-of-turn detection model to the specific scenario (a short order and a long consultation need different thresholds), tuning barge-in so the agent yields the floor rather than talking over you, building a latency budget across the whole pipeline, and testing all of it on real — not "convenient" — utterances: with pauses, dictated digits, background noise. It's dull, invisible work done before we ship — and it's what decides whether the agent sounds like a person or like a robot with a nice voice.
Where to start
- Measure the rhythm, not the timbre. On a test, listen to the pauses: does the agent interrupt, does dead silence hang, is the moment it takes the floor natural.
- Don't tune a single silence slider. A short threshold = false interruptions; a long one = sluggishness. Give the agent semantic end-of-turn detection instead.
- Test on "inconvenient" utterances. Dictating a number with pauses, "umm," re-asking, background noise — that's exactly where VAD breaks.
- Check barge-in separately. Interrupt the agent mid-sentence. It should go quiet almost immediately — and not be silenced by an "uh-huh."
- Count latency across the whole pipeline, not just "voice speed." The biggest delay is in time-to-first-token and in the end-of-turn decision.
Why this matters right now
In 2026 the voice is table stakes: a human timbre is available to everyone in a couple of clicks. The advantage comes not from a "better voice" but from timing discipline: an agent that predicts the end of your turn, yields the floor when you interrupt, and holds a rhythm close to the human 200–300 ms feels alive — even with a simpler voice. A "perfect" timbre with bad timing stays a robot. Whoever understood this earlier builds voice agents you want to finish the conversation with, instead of hanging up on.
