Ghosted Layers: Unconstrained Activation Alignment for Recovering Layer-Pruned LLMs

A team of researchers has introduced Ghosted Layers, a training-free module designed to recover performance lost when entire Transformer decoder blocks are removed from large language models, according to a preprint posted to arXiv [1]. Layer pruning eliminates complete decoder blocks from large language models to reduce computational cost, but the process creates a mismatch between the hidden state received by the next surviving layer and the distribution it was originally trained to process, causing significant performance degradation [1][2]. Daniel Yun and collaborators propose Ghosted Layers to address what they term a boundary activation alignment problem [1][2]. The method derives a closed-form optimal linear operator from a small calibration set to reconstruct the activation discrepancy introduced by the pruned layers [1][2]. The authors show that this solution corresponds to the unconstrained optimum of the alignment objective, whereas existing methods are restricted to constrained solutions over limited operator subspaces [2]. Large language models, typically built on transformer architectures, are pre-trained on vast text corpora to predict subsequent words and are later fine-tuned for tasks such as instruction-following and assistant-style interaction [8]. Pruning these models by removing layers can lower inference costs, but the abrupt change in internal representations often degrades output quality [1][2]. Ghosted Layers inserts a lightweight recovery module at the boundary where layers were removed, aligning activations without requiring additional training [1][2]. Experiments across multiple LLM backbones and pruning strategies demonstrated that Ghosted Layers consistently improves accuracy and perplexity over prior training-free baselines while preserving the efficiency gains of layer pruning [1][2]. The official code repository has been made publicly available on GitHub [2]. The preprint was submitted to arXiv on 15 May 2026 as version one, weighing 788 KB, and revised on 7 June 2026 as version two, weighing 789 KB [1]. arXiv, founded in 1991, is an open-access repository that hosts electronic preprints across disciplines including computer science, mathematics, and physics, and as of late 2024 receives approximately 24,000 new articles per month [6]. Submissions are moderated but not peer-reviewed [6]. The platform also supports community-built tools through its arXivLabs framework, which allows third-party collaborators to develop experimental features such as citation explorers and code-finding services that appear on article record pages [3][5].