Retrospective Progress-Aware Self-Refinement for LLM Agent Training

A new training framework called RePro aims to give large language model agents the ability to self-assess their progress on complex tasks, addressing a shortcoming that has hindered performance on long-horizon problems, according to a paper submitted to arXiv on 12 Jun 2026 [1]. The framework, formally named Retrospective Progress-Aware Training, targets a specific gap in current reinforcement learning-trained agents: they optimize step-by-step actions but lack metacognitive awareness of their own task progress [1]. The researchers' pilot study found that simply prompting an agent to report its progress online during a task actually hurt performance, while providing retrospective demonstrations of progress assessment was helpful [2]. However, the paper notes this capability cannot emerge from outcome-reward training alone [1]. To solve this, RePro uses a "forward-then-reflect" rollout paradigm. The agent first executes a task while generating online progress estimates; once the task is complete and the outcome is known, the agent retrospectively reassesses its step-wise progress, anchored by the final result [3]. The training process has two phases. The first, called Retrospection Warmup, uses a small set of external-LLM demonstrations to teach the agent the format for retrospective reflection [4]. The second phase, RePro-PO, uses a composite reward that combines retrospective progress shaping, online-retrospective alignment, and format regularization to produce per-step training signals that complement the sparse outcome reward [5]. The researchers tested RePro on three distinct tasks: WebShop, ALFWorld, and Sokoban [1]. When applied to models in the Qwen family, the framework delivered up to 12% absolute success rate gains [1]. The paper was submitted to arXiv, an open-access repository for electronic preprints that, as of November 2024, receives about 24,000 articles per month and hosts over two million papers across fields including computer science and mathematics [9].