The Curse and Blessing of Mean Bias in FP4-Quantized LLM Training

A research team has identified a mean bias as the primary source of instability in FP4-quantized large language model training and developed a hardware-efficient method called Averis to address it, according to a paper published on arXiv [1]. Training large language models with 4-bit floating point (FP4) precision offers substantial memory and compute savings but has remained fragile. The fragility stems from blockwise quantization being dictated by extreme activation magnitudes, which inflate dynamic range and compress long-tail signals [1]. The researchers found a counterintuitive source of this failure: dominant activation outliers are largely induced by a coherent rank-one mean bias, whose direction aligns with the leading anisotropic spectral component [1]. This mean component strengthens during training, is amplified and reshaped by attention and feed-forward network operators, and increasingly dominates top activation magnitudes [1]. The discovery revealed that a seemingly complex outlier-suppression problem admits a simple solution: isolate the coherent mean before quantization [1]. The team proposed Averis, a mean-residual splitting quantization method that separates the mean component using only reductions and elementwise subtractions before FP4 quantization [1]. Across Qwen3 0.6B Dense trained on 100 billion tokens and Qwen3 7B A1.5B MoE trained on 50 billion tokens, Averis enabled robust W4A4G4 FP4 training [1]. It reduced BF16 loss gaps to 1.19% and 0.81%, compared to 2.05% and 1.10% for NVIDIA's recently released Hadamard-based outlier-smoothing method [1]. Downstream gaps were limited to 0.89 and 0.71 points [1]. The method carries only a 2.20% end-to-end overhead over vanilla NVFP4, which is about 30% of the design complexity of NVIDIA's Hadamard-based approach [1]. When combined with the Hadamard method, Averis further reduced the Qwen3-0.6B loss and downstream gaps to 0.94% and 0.73 points [1]. The paper was submitted by Hengjie Cao and revised in June 2026 [1]. Code for the implementation has been made available at an anonymous repository [1].