Voice agents at scale.

The session

Voice agents at scale.

A production voice agent is a system of four models running in a tight loop: a speech-to-text (STT) model, a language model (LM), a text-to-speech (TTS) model, and an audio turn-taking model. The user speaks. STT transcribes. The LM reasons. The LM drafts a response. TTS speaks it. The turn-taking model decides when the user has stopped talking so the agent can respond. The round-trip latency budget is 800ms. Anything longer feels broken. The cost per call is a function of the model sizes and the call duration. The architecture is the difference between a voice agent that works and one that does not.

The latency budget breaks down roughly as: STT 200ms, LM 400ms, TTS 200ms. The turn-taking model is constant-time and negligible. The LM is the variable. A 70B model with 200 tokens of output is faster than a 7B model with 800 tokens of output, because the cost is in the output tokens, not the model size. The right optimization is to keep the LM's output short — short sentences, no filler, no over-explanation. The user wants an answer, not a monologue.

The unit economics in 2026: a typical 60-second voice call costs about $0.007 in model inference at current token prices. A human receptionist answering the same call costs $0.30-0.80 fully loaded. The 50-100x cost differential is what makes voice agents viable for inbound call volume that no human team could staff. The break-even for replacing a human is around 5-10 calls/day per agent. Below that, the human is cheaper. Above that, the agent is materially cheaper.

The failure modes are specific and predictable. The first is interruption handling: the user starts talking while the agent is talking, the agent does not stop, the user gets frustrated. The fix is end-of-turn detection with interruption: when the user starts talking, the agent stops within 200ms. The second is hold music: the agent puts the user on hold to "look something up" and the hold is 15 seconds. The fix is a fast lookup or an explicit escalation to a human with a clear handoff. The third is the agent not knowing when to stop: the agent keeps talking past the user's "thank you" and the user hangs up frustrated. The fix is a clear end-of-conversation detector that lets the agent say goodbye and disconnect.

The voice quality matters more than most operators expect. A 24kHz model with natural prosody outperforms a 16kHz model with robotic prosody on user satisfaction scores, even when the words are identical. The 24kHz model costs 50% more per call. The right answer is to use the 24kHz model for the agent's first response (the most-heard utterance), and a cheaper model for the rest. The audio fidelity is the difference between a voice agent that feels like a person and one that feels like a robot.

The compliance picture is shifting. The FCC in the US and the CRTC in Canada have rules about AI-generated voice calls (the TCPA, the AITC, the Canadian Anti-Spam Legislation). The EU AI Act classifies voice agents as "limited risk" with transparency requirements. Operators in regulated industries (healthcare, finance) need a disclosure at the start of the call ("this call may be recorded and is being handled by an AI agent"), an opt-out path, and an escalation to a human at the user's request. The compliance work is not optional.

The Netwit voice agent stack (Voice Agently) is built on top of the same Agentic OS runtime. The voice pipeline is a sequence of MCP tools (STT, LM, TTS, turn-taking) orchestrated by a supervisor agent. The supervisor can escalate to a human, can hand off to a different agent (a billing agent, a sales agent, a triage agent), and can record the full conversation for the audit trail. The architecture is a worked example of multi-agent orchestration applied to a real-time, latency-sensitive workload.

The session walks through the four-model architecture, the latency budget, the unit economics, the failure modes, the compliance rules, and a live demo of a voice agent handling a real inbound call end-to-end. The recording and the worked example blueprint are sent to all registrants.