Latency budget
A latency budget is the total time allowed between a caller finishing speaking and hearing a reply, divided across every step in between.
A phone conversation has a narrow tolerance. Human speakers swap turns in roughly 200 milliseconds, and a gap beyond about a second reads as hesitation or a dropped line. Whatever happens between those two moments has to fit inside that.
The steps are cumulative: detecting that the turn ended, transcribing the audio, generating a reply, synthesising speech, and moving audio across the network in both directions. Each has a floor you cannot go below, so the budget is best thought of as a fixed sum being spent rather than a target to optimise toward — buying a better model for one step means finding the time somewhere else.
Related
- Turn detection — Turn detection is deciding when a caller has finished speaking and it is the agent’s turn to reply.
- STT (speech to text) — Speech to text turns spoken audio into written words, in real time for a live phone call.
- TTS (text to speech) — Text to speech turns written words into spoken audio, and is what gives a phone agent its voice.