EPTS: Elastic Post-Training Sparsity for Efficient Large Language Model Compression

A new compression framework called Elastic Post-Training Sparsity (EPTS) aims to make large language models more adaptable for deployment across different hardware by generating a single model that works at multiple sparsity levels, according to a paper submitted on 24 Jun 2026 [1]. The work addresses a limitation in current Post-Training Sparsity (PTS) methods, which typically require a separate, time-consuming optimization session for each desired sparsity level [1]. This single-sparsity approach complicates deployment when hardware capabilities vary, as adapting to a new sparsity requirement mandates a complete re-optimization process [2]. EPTS is designed as a unified Multi-Sparsity framework that produces a single elastic model through a one-shot optimization process [1]. The framework incorporates a Multi-Sparsity Hierarchy LoRA (MS-HiLoRA) mechanism to facilitate knowledge inheritance from low- to high-sparsity groups, mitigating competition for parameter reconstruction [2]. A second component, the Multi-Sparsity Feature Mixer (MSFM), enhances the model's adaptability to pruning perturbations by dynamically fusing feature representations of varying sparsity granularities [2]. The researchers tested EPTS on the LLaMA and OPT model families, reporting competitive performance compared to state-of-the-art methods like SparseGPT and Wanda [1]. The source code for the project has been made publicly available on GitHub [2]. The paper was submitted to arXiv on 24 Jun 2026 [1].