Efficient Agentic Reinforcement Learning with On-Policy Intrinsic Knowledge Boundary Enhancement

A new training method called AKBE aims to make large language model agents more efficient by teaching them when not to use external tools, according to a paper published on arXiv. The approach addresses a problem where reinforcement learning causes models to make redundant tool calls, blurring the line between their own knowledge and the need for outside information. The method, detailed in a paper submitted on May 26, 2026, is called Agentic Knowledge Boundary Enhancement (AKBE) [1]. The researchers identify a core issue with standard agentic reinforcement learning (RL) for LLM-based agents: the training process induces a growing number of unnecessary tool calls and obscures the model's intrinsic knowledge boundary, which is the point at which a model should rely on its own parametric knowledge instead of querying an external tool [2]. Existing techniques to fix this problem often rely on reward shaping, which the authors argue creates coarse optimization targets. These targets can inadvertently push the model to suppress tool use entirely, a behavior the paper describes as reward hacking [2]. AKBE takes a different approach by dynamically probing the model's knowledge boundary during training. It does this by running dual-path rollouts for each question—one where the agent has access to tools and one where it does not—and then comparing the correctness of the results [2]. By analyzing these paired outcomes, AKBE categorizes the agent's trajectories and generates targeted supervisory signals. These signals are designed to guide the model toward efficient tool-use patterns on a per-instance basis, defining the knowledge boundary as whether tools are required and, if so, the minimum number of calls necessary [2]. The signals are then integrated directly into the agentic RL training loop [1]. The researchers tested AKBE across seven question-answering benchmarks. The results showed an average improvement in task accuracy of +1.85 and an 18% reduction in tool calls compared to standard agentic RL, leading to a 25% increase in what the paper terms tool productivity [2]. The authors note this gain comes without a trade-off between accuracy and efficiency [1]. Further analysis indicates the method is compatible with different RL algorithms in a plug-and-play fashion [2]. The code for AKBE has been made publicly available on GitHub [2].