Improving Code-Switching ASR with Code-Mixing Guided Synthetic Speech

A new method for generating synthetic speech aims to improve automatic speech recognition for code-switching by enforcing language-boundary consistency, according to research published on arXiv. The approach uses a preference-learning framework guided by the Code Mixing Index to produce training data that better reflects natural bilingual speech patterns [1]. Code-switching automatic speech recognition remains difficult because high-quality text-speech pairs for training are scarce [1]. While synthetic data augmentation through text-to-speech has been explored, existing systems prioritize reconstruction fidelity and do not explicitly enforce consistency at language boundaries, limiting their usefulness for code-switching ASR [1]. The proposed framework addresses this gap by steering synthetic speech generation toward improved code-switching fidelity using the Code Mixing Index, a metric that quantifies the degree of language mixing in an utterance [1]. Experiments were conducted on the SEAME Mandarin-English conversational corpus [1]. When the synthetic data was used to fine-tune the Whisper Large model, the Mixed Error Rate on the DevMAN set fell from 12.1% to 8.9%, and on the DevSGE set from 17.8% to 14.2% [1]. The results indicate that the preference-learning approach enhances the utility of synthetic data for ASR fine-tuning compared to conventional text-to-speech augmentation [1]. The challenge of limited training data is not unique to speech processing. In computational catalysis, researchers have explored transfer learning and joint training across datasets to improve model performance when individual datasets are small [5]. For instance, models trained on the OC20 dataset have been used to aid learning on the OC22 dataset, demonstrating that data from related tasks can be complementary [5]. The code-switching ASR work applies a similar principle by generating synthetic data that is explicitly designed to complement real recordings in its language-mixing characteristics [1]. The broader context of machine learning research continues to see rapid publication activity. A Wikipedia summary of scientific events in early 2023 noted advances across multiple fields, though it did not detail specific speech-recognition milestones [3]. The current work, submitted in June 2026, builds on years of incremental progress in both ASR and synthetic data generation [1].