Representation Forcing for Bottleneck-Free Unified Multimodal Models

Researchers have proposed new techniques to improve unified multimodal models (UMMs), which integrate understanding and generation within a single framework.

A technique called Representation Forcing (RF) has been proposed to eliminate the structural bottleneck in UMMs, which currently rely on a frozen, separately pretrained VAE for image generation^[2]. Removing this bottleneck introduces a quality gap, as the model must learn both high-level structure and low-level details from raw pixels. RF forces the decoder to autoregressively predict visual representations as intermediate tokens before pixels, eliminating the need for any external generative latent space^[2]. According to the researchers, RF benefits both understanding and generation in UMMs, and a pixel-space model with RF matches state-of-the-art VAE-based unified models on image generation^[2]. Another research effort has highlighted an 'understanding-generation gap' in UMMs, where models can capture user intent but fail to translate this semantic knowledge into precise pixel-level manipulation^[1]. This gap results in two bottlenecks in the anything-to-image task (X2I): the attention entanglement bottleneck and the visual refinement bottleneck. To address this, a novel framework has been proposed to empower unified models to autonomously switch between generation strategies based on instruction complexity and model capability. A high-quality dataset with over 50,000 samples has been contributed, and a two-stage training strategy comprising SFT and RL has been implemented^[1].