Vocabulary Dropout for Curriculum Diversity in LLM Co-Evolution

A new technique called vocabulary dropout can prevent language models from collapsing into repetitive patterns during autonomous self-play training, researchers report. The method applies a random mask to a model's output logits, sustaining diversity and yielding measurable solver improvements on mathematical reasoning tasks. The approach targets a known failure mode in co-evolutionary self-play, where one language model generates problems and a second model solves them. Without intervention, the proposer model quickly converges to a narrow distribution of problems that satisfy the reward function, a phenomenon the authors describe as diversity collapse [1][2]. This collapse renders the curriculum uninformative for the solver, stalling the co-evolutionary loop [2]. Jacob Dineen and collaborators introduce vocabulary dropout as a lightweight countermeasure. The method is a hard, non-stationary mask applied to the proposer's output logits during both policy training and curriculum generation, preventing the model from locking into fixed token sequences [1][2]. The researchers trained Qwen3-4B and Qwen3-8B models on mathematical reasoning using the R-Zero framework [1][2]. Across lexical, semantic, and functional metrics, vocabulary dropout sustained proposer diversity throughout training [1][2]. The diversity translated into solver performance gains averaging +4.4 points for the 8-billion-parameter model, with the largest improvements observed on competition-level benchmarks [1][2]. The paper was submitted on 3 April 2026 and revised most recently on 14 June 2026 [1]. The findings arrive amid broader efforts to make language model training more autonomous and cost-efficient. Large language models are typically trained with self-supervised learning on vast text corpora [7]. Recent industry milestones include DeepSeek's R1 model, which demonstrated comparable performance to leading models at a fraction of the training cost, reportedly using approximately one-tenth the computing power consumed by Meta's comparable Llama 3.1 model [6]. The vocabulary dropout paper is available on arXiv, which since 2022 has integrated with Hugging Face Spaces to allow researchers to attach interactive demos directly to paper abstract pages [3][4]. Authors can link demos by including a paper citation in a Space's README file or by associating a model on the Hugging Face Hub with the Space [5]. The integration covers papers in computer science, statistics, and electrical engineering and systems science categories [4]. The authors frame vocabulary dropout as one instantiation of a broader principle: explicit action-space constraints, analogous to the structural role that game rules play in classical self-play, can help sustain productive co-evolution in language [1][2].