Improved Baselines with Representation Autoencoders

Researchers have proposed an improved version of Representation Autoencoders (RAE) called RAEv2, achieving state-of-the-art results in image generation tasks with faster convergence. RAEv2 uses a generalized formulation and exhibits complementary working mechanisms with Representation Alignment (REPA)^[1].

RAEv2 achieves a state-of-the-art gFID of 1.06 in just 80 epochs on ImageNet-256, significantly improving upon the original RAE. It attains an EPFID@2 of 35 epochs, compared to 177 for the original RAE^[1]. Meanwhile, a new method called Drift-RAE has been proposed to distill pretrained flow models in RAE latent spaces using Drifting, achieving better results than state-of-the-art RAE distillation methods with 1.77 FID on ImageNet 256 dataset using only 10k distillation steps^[2]. Representation Autoencoders (RAEs) have been shown to improve diffusion and flow models with semantically richer latent space. However, the rich semantic representations can cause severe anisotropy and large curvatures, hindering convergence and performance in the distillation stage. Drift-RAE addresses this issue, but may not be suitable for extremely scattered spaces like reconstruction-based VAEs^[2].