Offline-to-Online Learning in Linear Bandits

A new algorithm for stochastic linear bandits aims to resolve a long-standing tension between relying on historical data and exploring new options, according to a preprint posted to arXiv on 3 June 2026 [1]. The method shifts its reliance from offline records toward active exploration as the number of online interactions grows. The work addresses a setting common in recommendation systems and personalized healthcare, where a learner must choose actions sequentially while receiving noisy rewards and also holds a batch of previously collected observations [3][5]. Purely offline approaches typically use a principle called pessimism in the face of uncertainty, which keeps decisions close to what the historical data supports but can incur large regret when the interaction horizon is long [3]. Purely online strategies, by contrast, rely on optimism to drive exploration and achieve sublinear regret over time, yet they may explore wastefully when the horizon is short [3]. The central challenge is therefore to blend these two principles, trading off between exploiting the offline solution and exploring to improve long-term performance [3]. The proposed algorithm manages this balance through an exploration budget. It selects the pessimistic action computed from the offline data during early rounds, then progressively increases exploration as the horizon extends [3]. The authors provide regret bounds showing the method is simultaneously competitive with both purely online and purely offline baselines. Regret relative to the optimal action grows sublinearly in the number of online interactions, while regret relative to an offline reference shrinks as the number of offline samples increases [1][2]. Empirical tests across varied problem parameters further demonstrated the approach’s effectiveness [2][4]. Related research underscores the practical importance of the offline-to-online setting. A separate 2025 study introduced the Offline-Online Phased Elimination algorithm, which uses an extended D-optimal design within each exploration phase to incorporate offline data and reduce online regret [5]. That work showed that when offline data is abundant and well-explored, online regret can drop substantially, and it provided the first minimax lower bounds that depend explicitly on offline-data quality [5]. The broader field of online machine learning, where models update sequentially as new data arrives, already supports applications such as sponsored search, real-time fraud detection, and dynamic pricing [7]. The new preprint extends this line of inquiry into structured linear environments where the offline-to-online tradeoff had remained poorly understood [1][3].