Real-Time Voice AI Hears but Does Not Listen

Four leading real-time voice AI systems from OpenAI, Google, and Alibaba consistently ignore vocal emotional cues during consequential decisions, even when they correctly perceive distress, fear, or sarcasm, according to a study submitted in 2026 [1][2]. The study evaluated OpenAI's GPT Realtime 2, Google's Gemini 3.1 Flash Live, and Alibaba's Qwen3.5 Omni Plus and Omni Flash across three scenarios where both words and vocal delivery carried meaningful information [1][2]. In one test, the systems ended calls with crying callers who insisted nothing was wrong. In another, they approved wire transfers authorized in frightened voices. They also enrolled callers whose agreement was clearly sarcastic [1][2]. Researchers found the failures were often not rooted in perception. When asked directly, three of the four systems reliably identified the distress, fear, or sarcasm they later ignored when making decisions [1][2]. The paper terms this disconnect the "emotional intelligence gap of voice AI" [1][2]. A similar pattern emerged when the systems estimated accent and age: responses frequently followed the biases of the words rather than the acoustic properties of the speaker [1][2]. Prompting the systems to explicitly attend to vocal delivery improved performance only partially and inconsistently [1][2]. The authors concluded that current real-time voice AI systems often behave as if speech had been reduced to a transcript, and urged caution in settings where tone and emotion convey important information [1][2]. Virtual assistants have grown rapidly in capability and reach. Such technologies often incorporate chatbot capabilities to streamline task execution, and interaction may be via text, graphical interface, or voice [6]. Into the 2020s, the emergence of AI-based chatbots brought increased capability and interest to the field of virtual assistant products and services [6]. The study's findings suggest that even as these systems grow more sophisticated in generating and recognizing speech, a gap remains between acoustic perception and downstream action. Speech synthesis, the artificial production of human speech, has advanced considerably since the earliest computer operating system to include a speech synthesizer — Unix in 1974 — and the default text-to-speech voice Microsoft Sam shipped with Windows 2000 [3]. Modern systems can incorporate models of the vocal tract and other human voice characteristics to create synthetic voice output [3]. Yet the new research indicates that real-time voice AI, which combines speech recognition and synthesis with decision-making, has not closed the loop between hearing emotional signals and acting on them [1][2].