Near-Optimal Stochastic Linear Bandits with Delay

Researchers have made advancements in understanding stochastic linear bandits with delayed feedback, establishing near-optimal regret guarantees and proposing a new algorithm, SME-OFU, for stochastic linear contextual bandits.

A study on stochastic linear bandits with delayed feedback has identified conditions under which these bandits exhibit similar behavior to multi-armed bandits (MAB) and when they present new challenges^[1]. The research found that under loss-independent delays, the regret penalty is additive and scales with the expected delay under stochastic delays and with the maximum number of outstanding observations under adversarial delays^[1]. In contrast, linear bandits are substantially harder than MAB under loss-dependent delays, with the optimal MAB guarantee unattainable in linear bandits under the delay-as-payoff model^[1]. Meanwhile, a new algorithm, SME-OFU, has been proposed for stochastic linear contextual bandits with bounded noise, achieving an improved regret bound of O(log T) compared to the existing optimal bound of O(sqrt(T)) for sub-Gaussian noise^[2]. Simulations have shown empirical improvements of SME-OFU over a benchmark algorithm designed for sub-Gaussian noise^[2].