LaGO: Latent Action Guidance for Online Reinforcement Learning

A new framework called LaGO uses large language models to softly guide online reinforcement learning, improving policy optimization without requiring the LLM to act as a direct controller, according to research posted to arXiv on June 23 [1]. The method, Latent Action Guidance for Online Reinforcement Learning, treats a pretrained LLM as a latent action prior rather than an explicit planner. Prior work often relied on LLMs as direct controllers, which demands precise action generation and can prove unreliable in practice [2]. LaGO instead integrates the model’s knowledge into the policy optimization loop, guiding the agent without dictating every action. Researchers tested LaGO against Vanilla PPO on two benchmarks. On the discrete-control CLEVR-Robot benchmark, the average success rate rose from 15.1% to 27.2%. On the continuous-control Meta-World benchmark, the rate climbed from 2.7% to 15.2% [1][2]. The framework consistently improved both reward and success rate across tasks. The paper’s analysis indicates that stronger pretrained LLMs deliver more effective guidance, suggesting that the embedded knowledge within these models can improve planning and online decision-making [1][2]. This aligns with broader trends in artificial intelligence research, where terms such as reinforcement learning, planning, and decision-making are central to the field’s subdisciplines [5]. While LaGO operates in a robotics and control context, the challenge of filtering and selecting optimal actions from a large space mirrors problems addressed by recommender systems. Those systems, widely used by social media platforms and streaming services, employ machine learning to analyze behavior and suggest relevant items from vast catalogs [3]. The LaGO framework similarly narrows the action space by using a prior, though its domain is physical task execution rather than content recommendation. The study also touches on concepts studied in psychology, which examines behavior, decision-making, and mental processes such as motivation and cognition [4]. By using a pretrained model to shape an agent’s policy, LaGO introduces a form of learned behavioral guidance that parallels how prior knowledge influences decision-making in biological systems. The work does not claim to model human psychology but operates in an adjacent space where artificial agents learn sequential tasks. The research contributes to ongoing efforts to combine the representational power of large language models with the adaptability of online reinforcement learning. The authors frame LaGO as a way to harness LLM knowledge without the brittleness of direct action generation, offering a path toward more reliable autonomous agents [1][2].