Motion-Compensated Weight Compression

A new weight-compression codec borrows a core idea from video encoding—motion compensation—to shrink neural networks by exploiting redundancy across layers rather than treating each layer in isolation [1][2]. The method, called Motion-Compensated Weight Compression (MCWC), was detailed in a paper submitted to arXiv on 23 May 2026 by Dr. Ismail Lamaakal [1]. It targets a growing deployment bottleneck: the sheer size of neural network weights [2]. Most existing compression pipelines process layers independently, missing cross-layer redundancy that arises from function-preserving symmetries [1][2]. MCWC addresses this by first aligning permutation-symmetric blocks, such as hidden units and attention heads, to maximize correspondence from one layer to the next [1][2]. This alignment step effectively turns network depth into a predictable sequence, much like consecutive frames in a video [2]. The technique draws its name and central mechanism from motion compensation, a foundational algorithm in video compression standards such as MPEG-2 and H.26x [3]. In video, motion compensation predicts a frame by describing how a reference frame has transformed, reducing the data needed to store the sequence [3]. MCWC applies a similar principle to weight tensors. After alignment, a lightweight layer-sequential predictor forecasts the weights of subsequent layers, with periodic keyframes inserted to reset prediction drift [1][2]. Only the quantized prediction residuals—the difference between the predicted and actual weights—are encoded using a learned entropy model trained under a rate-distortion objective [1][2]. Decoding is designed for speed. A simple decoder reconstructs deployable weights through four steps: entropy decoding, dequantization, predictor-driven reconstruction, and inverse alignment [1][2]. The authors report that this pipeline enables fast weight materialization for inference [2]. Across Transformer-based language modeling and vision classification tasks, MCWC improved the rate-accuracy Pareto frontier over strong quantization baselines and other learned weight-codec methods, while maintaining competitive decode time [1][2]. Ablation studies confirmed that each component—alignment, prediction, entropy modeling, and keyframe scheduling—was necessary to achieve the full compression gains [1][2]. Code for the project is available on GitHub [2].