Accelerating Hierarchical Sparse Predictive Coding with Hybrid Amortized Inference

A hybrid inference method that pairs a fast amortized encoder with a short corrective recurrence can accelerate hierarchical sparse predictive coding, according to a preprint posted to arXiv on 26 June 2026. The approach outperforms pure amortization while running much faster than classical iterative inference [1][2]. Hierarchical predictive coding frames perception as error-driven inference across stacked generative models, and adding explicit sparsity constraints yields representations with both computational and neuroscientific appeal [1][2]. In practice, however, extracting a useful sparse code for each input has required many recurrent refinement steps, a cost that grows as the hierarchy deepens [1][2]. Kazuhisa Fujita and collaborators examined this bottleneck by keeping the hierarchical sparse energy function fixed and varying only the inference procedure [1][2]. They compared four schemes: classical iterative inference based on the ISTA algorithm, an accelerated MFISTA reference, structurally informed amortized inference using a LISTA-style bottom-up encoder adapted to the hierarchical model, and a hybrid method that takes the fast amortized initialization and follows it with a small number of corrective energy-based refinement steps [1][2]. The evaluation measured reconstruction quality, sparsity, latency, and stability on static image benchmarks [1][2]. The results indicate that a shallow LISTA-style initializer plus short corrective recurrence improves over pure amortization while remaining much faster than long iterative inference [1][2]. The preprint was submitted as a 175 KB PDF and had not been peer-reviewed at the time of posting [1][6]. arXiv, founded in 1991, is an open-access repository that hosts e-prints across physics, mathematics, computer science, and related fields after moderation but without formal peer review [6]. As of November 2024, the repository was receiving roughly 24,000 new articles per month [6].