Plan Before Search: Search Agents Need Plan

A team of researchers has proposed a self-bootstrapping method for training retrieval-augmented reasoning agents, eliminating the need for distillation from a stronger external model. The approach, detailed in a paper submitted to arXiv on 27 May 2026, centers on a structured behavior called Plan that decomposes multi-hop questions before retrieval begins [1][2]. The paradigm combines reinforcement learning with Plan, which breaks a question into ordered sub-questions so each search step is anchored to a pre-designed query rather than drifting under the influence of partially relevant documents retrieved earlier [1][2]. The authors tested the method across three model families spanning 3B to 14B parameters and found that an identical reward signal induced qualitatively different reinforcement-learning failure modes depending on the model [2]. This observation led them to conclude that successful training depends not only on reward design but also on model-specific feasibility conditions, including sufficient initial entropy, training stability, and prerequisite sub-skills [2]. Motivated by these findings, the team developed a self-bootstrapping pipeline in which a small-scale seed model generates filtered trajectories that activate Plan in any target model, removing the reliance on an external stronger model for distillation [1][2]. The pipeline activated Plan across every tested model and consistently outperformed competitive baselines on multi-hop question-answering benchmarks [2]. The work was conducted within arXivLabs, a framework that allows collaborators to develop and share new arXiv features directly on the platform [1]. The paper was submitted to the Artificial Intelligence section of arXiv on 27 May 2026 [1].