LLMZero: Discovering Adaptive Training Strategies for RL Post-Training via LLM Agents

A new system called LLMZero uses large language model agents to automatically discover adaptive training strategies for reinforcement learning post-training, according to a preprint posted to arXiv on June 16, 2026 [1]. The system employs tree search to diagnose training pathologies and propose coordinated parameter adjustments across multiple stages [2]. The work, submitted to the machine learning section of the open-access repository, addresses a known limitation in reinforcement learning (RL) post-training: fixed training schedules cannot adapt to the non-stationary dynamics that emerge during multi-stage optimization [2]. Large language models, which are trained with self-supervised learning on vast amounts of text, have recently been applied to a range of meta-learning problems, and LLMZero extends this trend into training-strategy discovery [8]. The researchers identify a recurring empirical pattern across RL post-training tasks. Capacity parameters accumulate monotonically across stages, while regularization parameters predominantly oscillate in response to shifting training dynamics [2]. Fixed schedules commit all parameters to predetermined trajectories and therefore cannot express the exploration-exploitation tradeoffs that regularization must track, the authors write [2]. LLMZero addresses this by deploying LLM agents that perform tree search over training trajectories. At each checkpoint, the agents diagnose pathologies and propose coordinated multi-parameter transitions [2]. The approach was evaluated across four diverse GRPO tasks. LLMZero’s discovered strategies improved over the base model by 9% to 140% relative and over grid search by 6% to 15% relative, consistently outperforming both random search and a skill-based agent [2]. The structural principle uncovered by the system transferred across tasks, providing an explanation for why the discovered strategies take qualitatively different forms yet share similar parameter dynamics [2]. The authors argue this principle provides actionable design rules for multi-stage training [2]. The preprint appears on arXiv, an open-access repository of electronic preprints and postprints that, as of November 2024, receives about 24,000 submissions per month [6]. arXiv hosts papers across mathematics, physics, computer science, and related fields, and has surpassed two million articles as of late 2021 [6]. The repository is moderated but does not conduct peer review [6].