InfoPO: Information-Driven Policy Optimization for User-Centric Agents

A new training method called Information-Driven Policy Optimization, or InfoPO, treats multi-turn interactions between AI agents and users as a process of reducing uncertainty, rewarding the agent for turns that measurably sharpen its next actions [1]. The approach, detailed in a paper revised in June 2026, addresses a core weakness in current reinforcement-learning techniques for large language model agents. Existing methods such as those based on Group Relative Policy Optimization often compute rewards at the trajectory level, which the authors say leads to credit assignment problems and weak advantage signals within rollout groups [1][2]. InfoPO instead computes an information-gain reward at each turn by comparing the agent's action distribution against a counterfactual where user feedback was masked. This signal is then combined with task outcomes through an adaptive variance-gated fusion mechanism to maintain goal direction while identifying which exchanges carried the most information [1][2]. The paper, authored by Fanqi Kong and collaborators, tested InfoPO across tasks including intent clarification, collaborative coding, and tool-augmented decision making. It consistently outperformed both prompting and multi-turn reinforcement-learning baselines, and the authors report that it remained robust when user simulators were shifted and generalized to environment-interactive tasks [1][2]. The initial submission, dated February 28, 2026, was a 1,337 KB file; the revised version uploaded on June 17, 2026, expanded to 18,385 KB [1]. The work lands as India's AI market is projected to reach $8 billion by 2025, growing at a compound annual rate of 40 percent from 2020 to 2025, according to industry estimates cited by Wikipedia [3]. Government initiatives such as NITI Aayog's 2018 National Strategy for Artificial Intelligence have supported research at institutions including the Indian Statistical Institute and the Indian Institute of Science [3]. NASSCOM and Boston Consulting Group estimate that India's AI services could be valued at $17 billion by 2027 [3]. InfoPO's focus on fine-grained turn valuation departs from trajectory-level reward schemes that have been standard in multi-turn agent training [1][2]. The method's counterfactual comparison — measuring how much a user's feedback changes the agent's predicted next action — provides a per-turn credit signal that the authors argue is more targeted for learning effective collaboration [2]. The code repository is publicly available on GitHub [1][2].