OpenAI Rebuilds Voice AI Around Full-Duplex Conversation

OpenAI’s third-generation voice system, GPT-Live, is designed to solve one of the hardest problems in conversational AI: deciding when the system should speak. People naturally exchange turns in fractions of a second, using pauses, rhythm, tone, and context. Earlier voice assistants instead relied on small “turn detector” models to guess whether a person had finished talking. If the detector acted too early, the assistant interrupted the user; if it waited too long, the conversation felt slow. The main language model could begin its work only after that separate detector had made a decision.

GPT-Live replaces this stop-and-start design with a full-duplex voice model that can listen and speak at the same time. Full-duplex matters because it makes the interaction behave more like a continuous conversation than a sequence of recorded messages. Audio can keep flowing into and out of the model without waiting for a separate component to declare that one turn has ended. The result is intended to be faster, more responsive, and better aligned with the natural timing of human speech.

Earlier voice systems largely inherited the turn structure of text-based language models. In a traditional cascaded pipeline, speech recognition first converted audio into text, an LLM generated an answer, and text-to-speech then turned that answer back into audio. Running those stages one after another introduced delay and discarded expressive information such as pacing and tone. Direct speech-to-speech models improved matters by understanding and producing audio natively, preserving more of those signals and reducing processing time, but they still depended on a turn detector before inference could start.

GPT-Live instead places the voice model at the center of the conversation. Its most important responsibility is maintaining an uninterrupted loop in which incoming speech reaches the model and generated speech returns to the user. More demanding work—including deeper reasoning, tool calls, and consultation with frontier models such as GPT-5.5—runs asynchronously outside that live loop. This allows the system to seek a more capable answer or perform an action without freezing the conversation’s audio path while the secondary task is processed.

The architectural breakthrough is not merely a faster model; it is the separation of real-time media from slower application logic. Audio travels between the client and the voice model over a dedicated fast path. Tool use, delegation, policy handling, backend operations, and conversation persistence sit behind an asynchronous remote-procedure-call boundary. If a tool or backend service responds slowly, its own result is delayed, but that delay should not stop audio frames from continuing to move through the system.

This separation also creates a practical customization boundary. Developers can change an application’s tools, policies, or backend behavior without modifying the media frontend that protects conversational responsiveness. Keeping the live path small and predictable reduces the number of components capable of producing audible pauses. It also lets application features evolve independently from the latency-sensitive infrastructure responsible for moving speech.

Continuous audio introduces stricter engineering requirements than a turn-based exchange. A system processing separate audio blobs can tolerate some variation in when each blob arrives. A live media loop must deliver individual audio frames on schedule: delays in network transport, processing, or inference can immediately become noticeable pauses, glitches, or other artifacts. Responsiveness therefore has to be protected across the entire route, rather than improved in only the model or client.

OpenAI built on infrastructure previously developed for ChatGPT Voice and the Realtime API, which already streamed audio and video with lower and more predictable latency. GPT-Live extends that approach by streaming media all the way into the model through a new stateful inference system. “Stateful” means the inference layer maintains the ongoing conversational session instead of treating every user utterance as an isolated request. That continuity is essential for a model that must remain engaged throughout a flowing exchange.

Statefulness also adds production challenges. The platform must reliably transport audio from the client to the inference stack, preserve the relevant conversation context, and keep the session consistent while several activities occur concurrently. OpenAI says the six-month engineering effort required changes to model inference, context management, media transport, and delegation. The media frontend and inference logic were written in Go, replacing an earlier Python asyncio-based implementation, although the supplied source excerpt ends before explaining that migration in detail.

By combining a continuously active voice model with asynchronous access to stronger reasoning models and tools, GPT-Live aims to provide both conversational immediacy and deeper intelligence. The architecture now supports capabilities in ChatGPT Voice, including the newly launched ability to control a computer and coordinate agents through the ChatGPT desktop application. More broadly, it establishes a foundation for voice applications that can remain responsive while carrying out complex work in the background, bringing AI interaction closer to the fluid timing people expect from a live conversation.