Perplexity Can Miss SAE Feature Damage Under Quantization

A new study finds that perplexity scores can mask significant damage to the internal features of large language models after quantization, a common compression technique used for deployment. The research, led by Evan Duan, demonstrates that a quantized model can maintain or even improve its perplexity while a substantial fraction of its interpretable features, as measured by sparse autoencoders (SAEs), are degraded or destroyed [1]. The study used a frozen SAE as a fixed measurement basis to compare full-precision and round-to-nearest (RTN) quantized activations on identical tokens across models including Pythia-70M and Gemma-2-2B [2]. On Gemma-2-2B, INT7 quantization actually improved perplexity, yet 18.7% of active SAE features were degraded [1]. Under a sliding-window evaluation, INT6 also improved perplexity while only 51.3% of active features survived [2]. The paper describes this feature damage as "graded rather than cliff-like," with most non-surviving features being blurred instead of fully destroyed [1]. The survival rate for active Pythia features at INT6 was 62.4% [2]. The study also found that feature survival is highly predictable from full-precision feature statistics alone, achieving a cross-validated AUC between 0.92 and 0.97, with peak activation being the strongest marginal predictor [1]. Furthermore, the research showed that RTN quantization and magnitude pruning, when matched for perplexity, damage strongly overlapping sets of features, with a Jaccard overlap of 0.79 to 0.86 and a Spearman correlation of 0.98 for their damage scores [2]. These findings challenge the common industry practice of relying on behavioral metrics like perplexity as sufficient evidence for model integrity after compression. The broader context of AI model evaluation is already under scrutiny. A separate analysis of documentation for frontier models, including those from Anthropic and Microsoft, found systematic gaps in transparency, with safety-critical categories like deception behaviors and hallucinations accounting for the largest deficits across evaluated systems [3]. This documentation gap parallels the study's core warning: surface-level metrics can obscure deeper issues in model behavior and reliability. The research motivates feature-level audits for compressed models, arguing that full-precision interpretability findings cannot be assumed to transfer to their quantized counterparts without direct verification [1].