PROSE: Training-Free Egocentric Scene Registration with Vision-Language Models

A team of researchers has proposed PROSE, a training-free pipeline that uses a single pretrained vision-language model to register egocentric RGB captures of the same indoor space taken at different times, according to a paper submitted in 2026 [1]. The method, called Prompted Scene rEgistration, lifts each RGB sequence into an object-level 3D scene graph using off-the-shelf foundation models for geometry, segmentation, and language, then prompts the same VLM to match object instances across the two sequences [1]. To make the matching tractable, PROSE compares only objects at compatible heights and verifies each proposed match with a paired same/different query [2]. A rigid transform is then selected from per-pair candidates by inlier-ratio voting [2]. The pipeline adds no learned parameters and requires no depth sensor, training, or annotated graph [1]. On the Aria Digital Twin and Aria Everyday Activities benchmarks, PROSE outperformed both geometric and learned scene-graph baselines in registration accuracy, on ground-truth and RGB-reconstructed point clouds alike [1]. The open-vocabulary scene graph it produces also transfers directly to downstream tasks, including a simulated path-planning task [2]. The work arrives amid broader efforts to fuse vision-language models with 3D egocentric perception. EgoSplat, a language-embedded 3D Gaussian Splatting framework, addressed challenges such as frequent occlusions and dynamic artifacts by introducing a multi-view instance semantics extraction mechanism and an instance-aware spatio-temporal transient prediction module [3]. Evaluated on two egocentric datasets, EgoSplat set a new benchmark for open-vocabulary localization and segmentation [3]. Separately, the ProFuse framework strengthened semantic coherence in 3D Gaussian Splatting by injecting cross-view consistency and intra-mask cohesion directly into a registration pipeline, without render-supervised training [4]. Other recent work has explored articulated 3D scene graphs from egocentric vision. The Pandora system leveraged egocentric data captured as a human explores a scene wearing Project Aria glasses to recover models of articulated object parts, integrating them into 3D scene graph representations to enhance a robot's ability to perform mobile manipulation tasks [5]. Meanwhile, the EgoProx benchmark was introduced to evaluate egocentric 3D proximity reasoning in multimodal large language models, revealing that limited spatial understanding arises not from missing spatial knowledge but from ineffective mechanisms for leveraging knowledge already encoded in model parameters [6].