Stage-1 Controls the Entropy Regime, Not the Outcome

A study of two-stage post-training for vision-language models finds that the first stage controls the entropy regime rather than the final outcome, with different warm-start methods converging to similar performance after reinforcement learning [1]. Researchers examined what Stage-1 warm-starts actually control in a small-data study using Qwen2.5-VL-7B with a same-modality 72B VLM teacher for on-policy distillation (OPD) [1]. Three warm-start methods—two supervised fine-tuning (SFT) variants and OPD—reached a narrow 53–54% band on Geometry3K internal validation, consistent with the narrow range reported by recent specialized methods [1]. This setup provided little evidence that Stage-1 changes the in-domain endpoint [1]. A matched-recipe, early-stopped SFT improved out-of-domain MathVista performance by +2.1 points, reversing the −9.5-point drop of an over-trained variant [1]. The clearest difference emerged in the entropy regime: OPD entered reinforcement learning (RL) with substantially higher policy entropy than either SFT initialization, and the separation remained visible through the available trajectories [1]. At the in-domain initialization, OPD also showed higher answer diversity and pass@16 improvements of +2.0 to +5.2 points over SFT, although problem-level bootstrap intervals indicated the smaller contrast was uncertain [1]. The advantage disappeared after RL, with endpoint pass@16 values within 1.1 points, and on MathVista six models fell within 1.2 points of each other [1]. The authors characterize the contribution as a bounded empirical finding: Stage-1 is strongly associated with the entropy regime in this setup, but the downstream payoff is small, localized, and does not constitute evidence that OPD is a better RL warm-start [1]. The concept of bias in the introduction of variation, formalized in evolutionary biology, describes how biases in the generation of heritable variation can influence outcomes through a first-come, first-served effect [5]. This theoretical framework parallels the study's observation that initial entropy differences do not necessarily translate into lasting performance advantages after selection pressures—in this case, RL optimization—are applied [1][5].