Self-EmoQ: Plutchik-Guided Value-based Planning to Drive Streaming Emotional TTS

A research team has introduced Self-EmoQ, a framework designed to equip conversational AI with the ability to determine its own emotional state before generating speech, aiming to make synthetic voices more contextually appropriate in real time. The system, detailed in a paper submitted to arXiv on 21 April 2026, addresses a gap in current conversational AI, which the authors say lacks a "self-emotion determination mechanism" to guide streaming text-to-speech synthesis [1][2]. Self-EmoQ operates by planning an emotion prior to generating the textual response, grounding the subsequent emotional TTS output in a streaming manner [1]. The framework is built as a plug-and-play module initialized from pretrained large language models and trained via reinforcement learning, where emotions serve as the actions [1][2]. A hybrid reward function combines imitation signals with theory-driven scoring based on Plutchik's wheel of emotions [1][2]. Plutchik's model, a foundational theory in affective science, organizes emotions into a circular structure of primary and blended states, providing a structured vocabulary that the framework uses to evaluate and select appropriate emotional responses [2]. The researchers tested Self-EmoQ on four established dialogue and emotion datasets: DailyDialog, EmoryNLP, IMEOCAP, and MELD [1][2]. Across these benchmarks, the method outperformed both prompting and finetuning baselines on metrics for emotion determination and overall response quality [1][2]. Beyond offline evaluation, the team implemented a complete streaming pipeline for real-time deployment. The resulting speech quality confirmed the framework's emotional alignment, contextual coherence, and expressive fluency, according to the paper [1][2]. Code, case studies, and demonstration materials have been made publicly available through a project page [1][2]. The work contributes to a broader push in human-computer interaction to make synthetic voices not just intelligible but emotionally resonant, a challenge that has drawn increasing attention as voice assistants and conversational agents proliferate in consumer and enterprise settings.