MUNI: Multimodal Unified Latent Diffusion for Coherent Any-to-Any Generation

A new multimodal generative framework called MUNI has been proposed that unifies any-to-any generation across different data types using a single end-to-end latent diffusion model, according to a paper submitted to the arXiv preprint server on 15 June 2026 [1]. The framework, detailed in a paper titled "MUNI: Multimodal Unified Latent Diffusion for Coherent Any-to-Any Generation," departs from the dominant approach of building multimodal models on large language model (LLM) architectures [1][2]. LLMs are machine learning models with many parameters trained on vast amounts of text for language generation tasks [8]. The authors argue that LLM-based multimodal systems limit the use of modality-specific generators and require text-paired data for training [2]. MUNI instead extends latent diffusion to handle any-to-any generation in an end-to-end fashion. Rather than following the standard two-stage recipe of precomputing a frozen latent space and then fitting a prior over it, MUNI jointly trains modality-specific encoders, expressive decoders, and a single shared flow-based prior under one objective [2]. The framework addresses what the researchers identify as a shortcoming in standard aggregation rules of multimodal variational inference when coupled with a learned prior and expressive decoders. They propose a routed training objective designed to align the shared latent with criteria of cross-modal coherence, predictive sufficiency of subset latents, and minimality of the latent content [2]. The paper reports experiments on the PolyMNIST-Quadrant-Labels benchmark and a large-scale image-text-audio benchmark. Results show MUNI matching or exceeding the strongest baselines on conditional generation while opening its largest margins on unconditional coherence [2]. The work appears on arXiv, an open-access repository of electronic preprints that is not peer-reviewed [6]. Founded in 1991, arXiv passed the two-million-article milestone by the end of 2021 and as of November 2024 receives about 24,000 submissions per month [6]. The paper's abstract page includes integrations from arXivLabs, a framework launched in 2020 that allows community collaborators to develop and share experimental tools directly on the site [4]. These tools, which appear as tabs below the abstract, include the Bibliographic Explorer for navigating citation trees and the CORE Recommender for discovering related open-access papers [5]. arXiv has stated that third-party collaborators under the arXivLabs framework have access only to minimal and anonymized user data, and any other use is prohibited without written consent [4].