How we built a realtime system for responsive voice AI in six months

OpenAI has detailed the development of GPT-Live, its third-generation voice AI system designed to deliver continuous, responsive voice interaction in six months. Previous voice AI architectures relied on turn detectors to guess when users finished speaking, introducing delays and awkward cutoffs. GPT-Live removes the turn detector from the audio path entirely, utilizing a full-duplex voice model that can listen and speak simultaneously for more natural conversations. To achieve low latency at scale, OpenAI reworked model inference, context management, and media transport. The system separates media flow from business logic by routing audio on a dedicated fast path, while delegation, tool use, and interactions with frontier models like GPT-5.5 occur asynchronously. This ensures that slow backend services or tool calls cannot stall the real-time audio loop. OpenAI implemented media frontend and inference logic in Go to improve frame delivery, adopting WebRTC to handle packet loss and clock drift. Additionally, the system manages operational tradeoffs like stateful inference and context compaction through a seamless handoff mechanism. By warming replacement model instances and prefilling context in parallel, GPT-Live can handle long-running calls and context compaction without interrupting the flow of conversation. The technology currently powers features in ChatGPT Voice, including computer control and agent coordination within the desktop app.