Gaussian Process Prior Variational Autoencoder for Endoscopic Videos

Researchers have introduced a Gaussian Process Prior Variational Autoencoder (GPVAE) framework to enhance endoscopic video restoration by leveraging temporal continuity to improve image quality and downstream tasks.

Endoscopic video analysis is crucial for gastrointestinal diagnosis and computer-assisted interventions. However, video sequences are often degraded by specular reflections, motion artifacts, and missing frames ^[1]. The GPVAE framework addresses this issue by combining endoscopy-specific encoders with scalable GP approximations: Hierarchical Prior Approximation (HPA) and Sparse Precision Approximation (SPA). According to the study published on arxiv.org, the GPVAE framework achieved a 21.9% reduction in image reconstruction RMSE and a maximum reduction of 26.1% relative to matched VAE baselines on the C3VDv2 colonoscopy dataset ^[1]. Furthermore, the framework reduced downstream trajectory RMSE by 12.7% on average across classical visual odometry and a pretrained PoseNet. Variational Autoencoders (VAEs), introduced over a decade ago, are widely adopted in research and industry for diverse applications, typically used as standalone models ^[2]. The novel GPVAE approach integrates VAEs as a neural network layer, providing per-frame uncertainty estimates that reflect temporal support and offer a confidence signal for restored frames. However, this comes at the cost of a 27.3% average increase in training time per epoch ^[1].