Joint Learning of Experiential Rules and Policies for Large Language Model Agents

A new framework called Joint Learning of Experiential Rules and Policies (JERP) aims to improve how large language model agents learn from past interactions by jointly updating a rule pool and the agent’s policy from the same experience trajectories, according to a paper submitted in 2026 [1]. The approach addresses a known limitation in building LLM agents for multi-step interactive environments. Prior methods either stored natural-language rules for prompting, which can fall out of sync with a changing policy, or used interaction data to update model parameters, which offers limited correction for local mistakes in sparse-reward settings [1]. JERP retrieves task-relevant rules at decision time and conditions the agent on them alongside interaction history. After each episode, the system uses the collected trajectories to optimize the policy and to revise the rule pool by comparing current rollouts with reference successful trajectories [1]. This coupling is designed to keep the rule pool aligned with the evolving policy while allowing stable behaviors to be gradually absorbed into the model itself [1]. The researchers tested JERP on AlfWorld and WebShop, two benchmark environments that require agents to complete sequences of actions to achieve a goal. The paper reports consistent gains in decision performance for complex interactive tasks [1]. The work builds on the broader challenge of making effective use of accumulated interaction experience, a problem that spans both rule-based and parameter-update strategies in reinforcement learning for language agents [2]. Simulation platforms such as AlfWorld and WebShop serve as imitative representations of real-world processes, allowing experimentation with agent behaviors under controlled conditions [5]. In such simulations, a model represents the key characteristics of a system, while the simulation itself tracks the evolution of that model over time [5]. The JERP framework operates within this paradigm, using simulated trajectories to refine both explicit rules and the underlying policy. By jointly learning rules and policies, JERP attempts to bridge the gap between interpretable, explicit knowledge and the broader generalization that comes from updating model parameters. The paper’s authors argue that this dual mechanism provides a way to correct local errors while preserving the stability of effective behaviors, a balance that has been difficult to strike in prior work [1].