When RL Fails after SFT: Rejuvenating Model Plasticity for Robust SFT-to-RL Handoff

A standard pipeline for improving large language models can break down when supervised fine-tuning is pushed too far, according to a paper submitted in 2026. Researchers found that excessive SFT robs models of the plasticity needed for subsequent reinforcement learning to be effective [1]. The work, posted to arXiv on 7 June 2026, examines the widely used sequence of Supervised Fine-Tuning followed by Reinforcement Learning for LLM post-training. The authors report that checkpoints subjected to excessive SFT often show limited improvement during RL, a failure they attribute to a loss of model plasticity — the reduced ability of an SFT-initialized policy to be reshaped by later optimization [1][2]. Their analysis, spanning parameter changes, output spaces, and RL optimization dynamics, indicates that over-trained SFT models produce over-confident token distributions and exhibit sharp parameter landscapes that resist further tuning [2]. To address the handoff problem, the team proposes a method called Rejuvenation. The technique combines base-anchored model fusion, which curbs the drift caused by prolonged SFT, with targeted neuron reset to reduce model rigidity. The goal is to restore plasticity while retaining the behavioral priors that SFT is meant to instill [1][2]. In experiments covering math reasoning and agentic tasks, Rejuvenation consistently lifted RL performance on over-trained SFT models and improved generalization to out-of-distribution tasks [2]. The findings arrive as the field continues to debate optimal post-training recipes. While the paper does not include external commentary, the underlying dynamic — where a preparatory phase inadvertently constrains a later learning stage — echoes broader challenges in machine learning optimization. The authors frame the SFT-to-RL transition as a plasticity bottleneck, a concept that may inform how practitioners schedule fine-tuning and reinforcement phases in future LLM development cycles [1][2].