LAFP: Preserving Latent Action Structure in Latent Policy Learning via Flow Matching

A new method called Latent Action Flow Policy, or LAFP, aims to preserve the multimodal structure of actions learned from video during robot policy training, according to a preprint posted to arXiv on June 9 [1][2]. The approach targets a known weakness in latent policy learning, where robots first learn action representations from large unlabeled video datasets and then refine those representations with a small amount of real-world interaction data [2]. Existing systems typically use behavior cloning for the second step, which the authors state tends to collapse inherently multimodal action distributions into unimodal ones, degrading the pretrained latent action structure [1][2]. Flow matching, a generative modeling technique, offers a potential fix, but applying it directly introduces a misalignment between latent actions and physical actions during decoder training because of the stochastic nature of the learned policy [2]. To address this, the researchers introduce an inference-time interpolation mechanism designed to mitigate the stochasticity-induced misalignment [1][2]. In experiments on downstream imitation learning tasks, LAFP consistently outperformed prior methods, achieving up to a 10-15% improvement in success rate while incurring less than 1x additional inference overhead [1][2]. The work appears on arXiv, the open-access repository that hosts preprints across physics, computer science, and related fields and which, as of late 2024, receives about 24,000 new submissions per month [6]. The paper was submitted under the Computer Vision and Pattern Recognition category and is accessible through standard arXiv abstract pages, which also feature community-developed tools such as the Bibliographic Explorer and CORE Recommender under the arXivLabs framework [1][5].