Making the Most of Limited Data: Score-Aware Training for Text-to-Music Generation

A research team has proposed a training method for text-to-music generation that treats audio-caption alignment scores as a direct supervision signal, aiming to reduce reliance on massive proprietary datasets and industrial-scale computing [1]. The approach, detailed in a paper submitted to the ICME 2026 ATTM Grand Challenge Efficiency Track, is called score-aware training [1]. The authors note that current state-of-the-art text-to-music systems depend on enormous, private data collections and compute resources, obscuring whether performance gains come from architecture or sheer scale [2]. Their method instead repurposes low-scoring audio segments by routing them to high-noise training regimes using a CLAP-conditioned Beta noise timestep schedule, which acts as an implicit regularizer [2]. A segment-level filtering step removes the most misaligned examples, and a two-stage captioning procedure bridges the gap between verbose training captions and the concise prompts used at inference time [2]. A REPA auxiliary loss transfers structured semantic knowledge from pretrained CLAP and MuQ encoders without requiring additional data [2]. The system, built on a 450M-parameter FluxAudio model, ranked second across both tracks in the objective evaluation and third in the Efficiency Track in the final mean-opinion-score evaluation [2]. The broader field of generative AI has expanded rapidly since the AI boom of the 2020s, driven by advances in deep neural networks and large language models based on the transformer architecture [5]. Generative audio models are part of a landscape that also includes text-to-image systems such as DALL-E and Stable Diffusion, and text-to-video models like Veo and Sora [5]. Google DeepMind, a subsidiary of Alphabet, has developed its own text-to-music model, Lyria, alongside other generative tools such as the Gemini family of large language models and the Imagen text-to-image model [3]. The new score-aware training work appears on arXiv, an open-access repository that hosts electronic preprints across physics, computer science, and related fields and currently receives about 24,000 submissions per month [10].