Towards On-Policy Data Evolution for Visual-Native Multimodal Deep Search Agents

Researchers have proposed a new method called On-Policy Data Evolution (ODE) to improve the performance of multimodal deep search agents by reusing intermediate visual evidence and continuously refining training data, according to a paper published on arXiv [1]. The study, authored by Shijue Huang and colleagues, targets a core limitation in current systems designed for multimodal deep search, where an agent must chain together search, tool use, and visual reasoning [1]. The first bottleneck identified is that existing tool-use frameworks treat images as transient outputs, preventing later tools from reusing intermediate visual evidence. The second is that training data is typically built with fixed curation recipes that do not adapt to the agent's evolving capabilities [1][2]. To address this, the team introduced a visual-native agent harness built around an image bank reference protocol. This system registers every image returned by a tool as an addressable reference, making it available for reuse by subsequent tools in a task chain [2]. On top of this harness, the ODE framework operates a closed-loop data generator that refines itself across multiple rounds using rollouts from the policy being trained. This process ensures each round's data targets the specific skills the current policy has yet to master [1][2]. The framework supports the generation of both supervised fine-tuning data and policy-aware reinforcement learning tasks, covering the full training lifecycle of the agent [2]. The performance gains were measured across eight multimodal deep search benchmarks. The Qwen3-VL-8B agent's average score rose from 24.9% to 39.0% with ODE, a result that the paper notes surpasses the 37.9% achieved by Gemini-2.5 Pro in a standard agent-workflow setting. At a 30-billion-parameter scale, the average score increased from 30.6% to 41.5% [1][2]. Large language models, which underpin such agents, are neural networks trained on vast text corpora and are often fine-tuned for specific tasks [3]. The quality of their training data is critical, as biased or inaccurate data can reduce reliability [3]. The ODE approach directly confronts the challenge of producing high-quality training datasets, a process that is typically difficult and expensive due to the labor required for labeling [4]. The paper's further analyses validated the effectiveness of image-bank reuse, particularly on complex tasks requiring iterative visual refinement, and found that the rollout-feedback evolution yielded more grounded training traces than static synthesis methods [2].