DASH: Faster Shampoo via Batched Block Preconditioning and Efficient Inverse-Root Solvers

A team of researchers has proposed DASH, a faster implementation of the Shampoo optimizer that accelerates training steps by up to 5.6 times through batched block preconditioning and new inverse-root solvers, according to a paper posted on arXiv [1]. The work, titled "DASH: Faster Shampoo via Batched Block Preconditioning and Efficient Inverse-Root Solvers," was submitted to the preprint repository on 2 February 2026 and revised on 25 June 2026 [1]. Shampoo is an approximate second-order optimizer that has won the MLCommons AlgoPerf competition and produces models with lower activation outliers that are easier to compress, but its adoption has been limited by significant computational overhead [1][2]. Ionut-Vlad Modoranu and collaborators address this bottleneck with two techniques. The first stacks preconditioner blocks into 3D tensors to improve GPU utilization. The second introduces the Newton-DB iteration and Chebyshev polynomial approximations as faster methods for computing the inverse matrix roots that Shampoo requires [1][2]. The paper also provides what the authors describe as the first in-depth analysis of how matrix scaling affects Shampoo convergence [2]. The resulting implementation, called Distributed Accelerated Shampoo, or DASH, achieves up to 5.6 times faster optimizer steps compared to the well-optimized Distributed Shampoo baseline [1][2]. Among the tested methods, the Newton-DB approach attained the lowest validation perplexity per iteration [2]. The code has been made publicly available on GitHub [1]. The paper appears on arXiv, an open-access repository that hosts electronic preprints across mathematics, physics, computer science, and related fields [6]. Founded in 1991, arXiv passed the two-million-article milestone by the end of 2021 and now receives roughly 24,000 submissions per month [6]. The platform is not peer-reviewed; papers are approved after moderation [6]. The DASH manuscript was posted in the Machine Learning category and is available in two versions, with the initial submission sized at 74 KB and the revised version at 105 KB [1].