Masked Diffusion Decoding as $x$-Prediction Flow

Researchers have proposed a continuous decoding framework for masked diffusion language models that allows tokens to accumulate partial progress at each step, replacing the binary commit-or-mask regime that limits performance under tight decoding budgets. The work, detailed in a paper submitted to arXiv on 27 June 2026, targets a known weakness in masked diffusion language models (MDLMs). These models generate text by iteratively unmasking tokens, but their standard decoder forces each position into an all-or-nothing decision: a token is either committed to a single output or left fully masked, discarding any intermediate predictive information [1]. The authors argue this binary approach forces premature, irrevocable commitments that degrade output quality when the decoding budget is constrained [1]. To address this, the team reinterprets mask prediction as clean-state prediction, or x-prediction, and uses it to induce a continuous flow in the input embedding space [2]. Under this view, each token trajectory starts from the deterministic mask embedding — a state on which MDLMs are explicitly trained — and is iteratively moved toward the predicted clean state [3]. Unlike image diffusion models, which typically begin from stochastic Gaussian noise and apply a globally synchronous schedule, the proposed framework initializes from the mask embedding and adapts the schedule to the asymmetric, context-dependent nature of language generation [4][6]. The framework introduces a confidence-based asynchronous update mechanism. Rather than advancing all positions in lockstep, the diffusion progress is accumulated token-wise, allowing some tokens to commit early and inform others while less certain positions remain revisable [5]. A lightweight policy network governs this process, and its training is formulated as a reinforcement learning problem [1]. When applied to the pretrained LLaDA model, the continuous decoder reached 97% of its full performance on the HumanEval dataset while using only 25% of the standard decoding budget [1]. The method requires only a few hundred alignment-training steps to run on off-the-shelf MDLMs, according to the paper [3]. Diffusion models, which learn to reverse a process of adding noise to data, have seen widespread use in computer vision tasks such as image generation through systems like Stable Diffusion and DALL-E [6]. Their application to natural language processing, including text generation and summarization, is a more recent development [6]. The proposed continuous decoding framework represents an effort to make diffusion-based text generation more practical under computational constraints by preserving richer predictive information throughout the decoding process.