Don't Go Breaking My LLM: The Impact of Pruning Attention Layers on Explanation Faithfulness and Confidence Calibration

Pruning attention layers in large language models can erode their interpretability and reliability even when accuracy holds steady, according to a new study that examined five models across eight datasets [1]. The research, posted to arXiv on 23 June 2026, investigates how removing attention layers — the most computationally expensive components of transformer-based models — affects explanation faithfulness and confidence calibration [1]. Prior work established that up to 33% of attention layers can be pruned with minimal accuracy loss, making the technique a cost-effective compression strategy [1][2]. However, the authors note that the downstream effects on interpretability had not been systematically studied [2]. Across the five large language models and eight datasets tested, pruned models frequently maintained high accuracy scores [1]. Yet the study found that faithfulness and calibration often degraded, and the two metrics could fluctuate significantly even when accuracy remained stable [2]. This misalignment suggests that standard accuracy benchmarks alone may not capture the full picture of a compressed model's reliability [1]. The paper recommends that evaluation protocols for pruned models incorporate explainability and calibration metrics alongside traditional accuracy and efficiency measures [1][2]. The findings arrive as the machine-learning community grapples with broader questions about model compression and transfer learning. Recent work in other domains, such as catalyst informatics, has explored how models trained on one dataset can aid performance on another through transfer learning or joint training, underscoring the importance of understanding what information is preserved or lost when models are modified [4]. While the arXiv study focuses on language models, its implications extend to any field where compressed neural networks are deployed in high-stakes settings. The authors caution that pruning attention layers can affect interpretability and reliability in ways that accuracy and efficiency measures alone do not reveal [2].