Variable-Rate Deep Image Compression based on Low-Rank Adaptation by Progressive Learning

A new method for variable-rate deep image compression uses Low-Rank Adaptation (LoRA) to match the performance of multi-model systems while sharply reducing storage and training costs, according to research posted to arXiv on June 15 [1]. The paper, titled "Variable-Rate Deep Image Compression based on Low-Rank Adaptation by Progressive Learning," addresses a persistent challenge in deep image compression (DIC). While DIC techniques have surpassed traditional codecs, enabling efficient data transmission for streaming, medical imaging, and connected vehicles, most variable-rate approaches either deploy multiple separate networks or rely on a single model that incurs higher computational complexity [1]. The authors propose a progressive learning strategy that inserts a LoRA Rate-Adaptive Module (LoRAM) into existing DIC architectures [1]. LoRA is a parameter-efficient fine-tuning method that adapts large pre-trained models by training only a small set of additional weights. Neural networks, which underpin modern DIC systems, consist of layers of artificial neurons that process signals through weighted connections adjusted during training [2]. By confining updates to low-rank matrices, LoRA dramatically reduces the number of parameters that must be stored and optimized. The new work exploits a re-parameterized merging property of LoRA: after training, the adapter weights can be folded into the base model, meaning the method "does not introduce additional computational complexity during inference" [1]. Comprehensive experiments reported in the paper show that the LoRAM-equipped approach achieves competitive compression performance against methods that use multiple models. The efficiency gains are stark: the technique saves 99% in parameter storage, 90% in datasets, and 97% in training steps [1]. These figures underscore a shift toward more resource-efficient machine learning pipelines, a concern that has grown alongside the scale of modern models. Large language models, for instance, now contain hundreds of billions of parameters and are trained on vast text corpora [10], making parameter-efficient adaptation a critical area of research. The work appears on arXiv, an open-access repository that has hosted over two million e-prints since its founding in 1991 and currently receives roughly 24,000 submissions per month [8]. The platform also supports community-built tools through its arXivLabs framework, which allows third-party developers to create features such as citation explorers and code linkers that enhance the reading and discovery experience [6][7].