SPARK: Spatial Policy-driven Adaptive Reinforcement learning for Knowledge distillation

A new framework called SPARK aims to improve how compressed image-restoration networks learn from their larger counterparts by focusing training effort on the most difficult parts of an image, according to a preprint posted to arXiv on June 13, 2026 [1][2]. The method, detailed in a paper by Mohamed Jismy Aashik Rasool, addresses a persistent problem in deploying artificial intelligence on phones and embedded systems [1]. Low-bit quantization, a technique that shrinks neural networks to run on resource-constrained devices, introduces rounding noise that disproportionately damages high-frequency details like edges and fine textures [2]. Existing knowledge distillation (KD) methods, where a smaller "student" model learns from a larger "teacher," apply correction signals uniformly, ignoring that some image regions are harder to reconstruct than others [2]. SPARK, which stands for Spatial Policy-driven Adaptive Reinforcement Learning for Knowledge Distillation, uses a lightweight reinforcement learning policy network to adaptively allocate distillation effort [1][2]. At each training step, a module computes four signals: Laplacian variance, pixel variance, student reconstruction error, and the teacher-student knowledge gap [2]. These are fed into a compact policy CNN that produces a stochastic spatial weight map, modulating the KD loss during quantization-aware training (QAT) [2]. The framework is designed to be task-agnostic for image restoration, adds no cost at inference time, and integrates into existing QAT pipelines without architectural changes [1][2]. The preprint, which is hosted on the open-access repository arXiv, has not yet been peer-reviewed [4]. arXiv, launched in 1991, now receives about 24,000 submissions per month and serves as a primary distribution channel for research in computer science and related fields [4]. The field of machine learning, which underpins this work, relies on statistical algorithms that learn from data to perform tasks without explicit programming, with deep learning models now surpassing many earlier approaches [3]. Experiments on benchmark datasets showed SPARK consistently outperformed post-training quantization (PTQ), standard QAT, and state-of-the-art KD approaches across multiple student architectures, achieving reconstruction quality closest to the full-precision teacher under significant computational constraints [1][2].