LISA: Likelihood Score Alignment for Visual-condition Controllable Generation

A new regularization technique called LISA aims to make the widely used dual-branch architecture for visual-condition controllable generation train faster and produce more disentangled features, according to a paper submitted to arXiv in 2026 [1]. The dual-branch paradigm trains a side network to encode visual conditions and fuses its intermediate-layer features into a frozen pretrained main network [1]. The approach has delivered strong results, but the precise role of the side branch and its training efficiency have remained underexplored [1]. The authors of the new paper revisit the paradigm through the lens of score-based generative modeling, arguing that the main network preserves visual perceptual quality by providing a prior unconditional score, while the side network steers conditional control by implicitly contributing a likelihood score [1][2]. Guided by that framing, the researchers propose Likelihood Score Alignment, or LISA [1]. The method hooks features from a designated layer of the side network and projects them into a score latent space using a lightweight decoder composed of convolution, activation, and upsampling layers [3]. An approximated likelihood score target is then constructed, and the distance between the decoder’s output and that target is calculated as an additional regularization loss [3]. The side network and the lightweight decoder are jointly optimized with both the standard diffusion loss and the LISA regularization loss, while the main network’s parameters remain frozen [3]. Experiments spanning image and video tasks, multiple architectures, and both diffusion and flow models showed that LISA consistently accelerated training convergence and improved final synthetic results [1][5]. The regularization also encouraged the side network’s features to become more disentangled for conditional modeling, with what the authors describe as negligible additional training cost and zero extra inference cost because the auxiliary decoder is discarded after training [1][4]. The paper was made available on arXiv on 25 June 2026 and is also indexed on Hugging Face’s papers page, where the abstract notes that score-based generative modeling reveals side networks contribute likelihood scores to conditional control [1][5].