BayLing-Duplex: Native Full-Duplex Speech Dialogue with a Single Autoregressive LLM

A research team has introduced BayLing-Duplex, a speech language model that can listen and speak simultaneously without relying on an external turn-taking module, according to a paper posted to arXiv on June 12 [1]. The model is built as a single autoregressive large language model that decides internally when to listen, speak, and stop [1]. Unlike existing speech language models such as LLaMA-Omni and GLM-4-Voice, which remain turn-based and depend on an external Voice Activity Detection module to mark the end of a user's turn, BayLing-Duplex handles natural conversational phenomena including overlap, hesitation, and barge-in [2][3]. The design adds only a few special tokens to the standard vocabulary, allowing it to transfer across LLMs and reuse existing training and serving stacks with no architectural adaptation [4]. The researchers started from the public GLM-4-Voice checkpoint and fine-tuned the model with 400,000 full-duplex samples, followed by a lightweight Direct Preference Optimization stage [1][5]. On the InstructS2S-Eval benchmark, BayLing-Duplex reached 92% turn-taking success and 100% interruption success [1][3]. It also improved the speech-response score from 2.17 to 3.39 over Moshi, a prior speech dialogue model [2][4]. Beyond interactive metrics, the model matched or surpassed its turn-based counterpart on Llama Questions, Web Questions, and Alpaca-Eval, indicating that simultaneous listen-and-speak modeling does not degrade response quality [1][5]. The paper's authors note that real-time, full-duplex speech interaction is a key feature of next-generation spoken chatbots [2][3]. Code and models have been made available on GitHub [4].