Spectral Evolution-Guided Token Pruning in Multimodal Large Language Models

A training-free framework called Cross-Layer Spectral Evolution (CLSE) aims to accelerate Multimodal Large Language Models by pruning redundant visual tokens without harming cross-modal reasoning, according to a paper submitted to arXiv on 23 Jun 2026 [1]. Existing token pruning methods typically rely on single-layer signals such as attention scores or token similarities, which can overlook how visual representations transform across layers and may introduce positional bias in multimodal token sequences [1]. To address this, CLSE quantifies token importance by modeling cross-layer token dynamics in the frequency domain, tracking how representations evolve from high-frequency structural details to low-frequency semantic abstractions across Transformer layers [1]. Tokens exhibiting stronger spectral redistribution across layers are considered more semantically active and are preserved [1]. Multimodal Large Language Models, or MLLMs, combine vision and language processing within architectures that often use Transformer networks [3]. Transformers are a class of deep learning models that have been applied to fields including computer vision and natural language processing, producing results that in some cases surpass human expert performance [3]. The computational cost of processing large numbers of visual tokens in these models has driven interest in pruning techniques that reduce floating-point operations, key-value cache memory, and latency [1]. The researchers report that CLSE provides a stable importance criterion that mitigates positional bias [1]. Experiments on image and video benchmarks showed that CLSE achieves a superior trade-off between efficiency and accuracy under aggressive token reduction, reducing FLOPs, KV cache memory, and latency while maintaining competitive or improved performance across multiple MLLMs [1]. The framework requires no additional training, distinguishing it from approaches that rely on fine-tuning or transfer learning to adapt models to new tasks [1][5].