HeatKV: Head-tuned KV-cache Compression for Visual Autoregressive Modeling

A new compression technique for Visual Autoregressive models can double the memory efficiency of their key-value caches without degrading image quality, according to a paper posted to arXiv. The method, called HeatKV, tailors cache allocation to individual attention heads rather than applying a uniform policy. Visual Autoregressive (VAR) models produce high-quality images with low latency but carry a heavy memory burden, often consuming gigabytes of KV-cache memory per generated image [1][2]. HeatKV addresses this by profiling how each attention head attends to previously generated scales. A small offline calibration set is used to rank the heads by their attention scores, and a static pruning schedule is then built for a given memory budget [1][2]. Applied to the Infinity-2B model, HeatKV achieves a 2× higher compression ratio in memory allocation for the KV cache compared to existing methods [1][2]. The compressed model maintains similar or better image fidelity, prompt alignment, and human perception scores, setting a new state-of-the-art for VAR KV-cache compression [1][2]. The paper was submitted to arXiv on 14 May 2026 by Jonathan Cederlund and revised on 17 June 2026. The first submission weighed 28,790 KB; the second, 29,410 KB [1]. Code and a calibration script have been released on GitHub under the arm-research organization [2]. KV-cache compression has become a focal point as generative models scale. While HeatKV targets visual autoregressive architectures, the broader research community is grappling with memory constraints across modalities. A separate scoping review of quantum circuit generation systems, for instance, found that no reviewed system reports end-to-end evaluation on quantum hardware, underscoring how deployment-stage bottlenecks persist in adjacent fields [3]. Hugging Face, which indexes papers and links them to models, datasets, and interactive demos, has integrated with arXiv to embed community-built Spaces directly alongside paper abstracts [4][5]. The HeatKV paper is discoverable through the Hugging Face Daily Papers feed, where trending preprints are surfaced by community submissions [6].