LEPO: Latent Reasoning Policy Optimization for Large Language Models

A new framework called LEPO applies reinforcement learning directly to continuous latent representations inside large language models, a departure from methods that operate on discrete token sequences, according to a preprint posted to arXiv in April 2026 [1]. The paper, authored by Yuyan Zhou and colleagues, was submitted on 20 April 2026 and revised through 12 June 2026 [1]. It addresses a limitation in recent latent reasoning approaches: without stochastic sampling, these systems tend to collapse into deterministic inference and miss alternative reasoning paths [2]. To restore exploratory capacity, the researchers inject controllable randomness via the Gumbel-Softmax technique [2]. Building on that stochastic foundation, the team proposes Latent Reasoning Policy Optimization, or LEPO. During the rollout phase, LEPO preserves stochasticity so that diverse trajectories can be sampled. In the optimization phase, it constructs a unified gradient estimate that covers both the continuous latent representations and the discrete output tokens [2]. The authors report that LEPO significantly outperforms existing reinforcement-learning methods designed for discrete and latent reasoning [1][2]. The work appears on arXiv, an open-access repository that hosts preprints across physics, mathematics, computer science, and related fields [6]. As of late 2024, arXiv was receiving roughly 24,000 new submissions per month and had surpassed two million total articles by the end of 2021 [6]. Papers on the platform are moderated but not peer-reviewed, a model that has made it the primary distribution channel for early-stage research in many quantitative disciplines [6]. The LEPO preprint is accompanied by experimental tools integrated through arXivLabs, a framework that lets community collaborators build features directly on the site [4]. Among the linked services are the Bibliographic Explorer, which maps citation trees, and the CORE Recommender, which surfaces related open-access papers from a global network of repositories [5]. arXiv introduced the Labs framework in 2020 to formalize such collaborations while requiring partners to adhere to the repository’s values of openness, community, excellence, and user-data privacy [4]. Large language models, the class of systems LEPO targets, are machine-learning models with many parameters trained on vast text corpora through self-supervised learning [8]. The LEPO submission history lists file sizes of 1,181 KB for the first three versions and 1,172 KB for the fourth [1].