Latent Gaussian Splatting for 4D Panoptic Occupancy Tracking

Researchers have introduced Latent Gaussian Splatting (LaGS), a new method for 4D panoptic occupancy tracking that models 3D scene features as a sparse set of feature-bearing Gaussians, achieving state-of-the-art results on two major autonomous driving benchmarks [1][2]. The work, submitted to arXiv in February 2026 by Maximilian Luz, Rohit Mohan, Thomas Nürnberg, Yakov Miron, Daniele Cattaneo, and Abhinav Valada, addresses a gap in how robots perceive dynamic environments [1][2]. Existing methods typically either provide coarse geometric tracking through bounding boxes or detailed 3D occupancy estimates that lack explicit temporal association and instance-level reasoning [1]. LaGS tackles both dimensions simultaneously by fusing multi-view camera observations into 3D Gaussians that serve as a sparse, point-centric latent representation of the scene, then splatting the aggregated features onto a 3D voxel grid decoded by a mask-based segmentation head [2]. The approach enables spatially continuous, distance-weighted aggregation of multi-view features before decoding, and a hierarchical Gaussian representation further allows multi-scale reasoning by combining global context from coarse super-points with fine-grained detail from higher-resolution streams [1]. The authors evaluated LaGS on the Occ3D nuScenes and Waymo datasets, reporting state-of-the-art performance for 4D panoptic occupancy tracking [1][2]. The broader field of 4D panoptic occupancy tracking has seen parallel efforts. A separate group recently proposed TrackOcc, a camera-based method that processes image inputs in a streaming, end-to-end manner using 4D panoptic queries and a localization-aware loss to improve spatial accuracy [3][4]. That work introduced the Camera-based 4D Panoptic Occupancy Tracking task, jointly addressing occupancy panoptic segmentation and object tracking in both spatial and temporal domains [3]. The localization-aware loss added position supervision for reference points of 4D panoptic queries, yielding an increase of 3.5 in OccSTQ and more than a 40 percent improvement margin in OccAQ [3]. Occupancy reconstruction remains a challenge for vision-only systems due to sparse viewpoints, dynamic elements, and severe occlusions [5]. A framework called GS-Occ3D recently demonstrated that decomposing scenes into static background, ground, and dynamic objects enables tailored modeling strategies, with ground explicitly reconstructed as a dominant structural element to improve large-area consistency [5]. The LaGS code and models have been made publicly available [1][2].