ManiSplat: Manipulation Trajectory Synthesis from Monocular Video via Decoupled 3D Gaussian Splatting

A new framework called ManiSplat can reconstruct controllable 3D digital twins of robotic manipulation scenes from a single ego-view video, according to a paper submitted to the arXiv preprint repository on 9 Jun 2026 [1][2]. The method separates robots, objects, and backgrounds into independently editable components for simulation and policy training [2]. The paper, titled "ManiSplat: Manipulation Trajectory Synthesis from Monocular Video via Decoupled 3D Gaussian Splatting," addresses a persistent obstacle in computer vision and robotics: rebuilding dynamic, interactive 3D scenes from real-world footage [2]. While 3D Gaussian Splatting has produced high-fidelity static reconstructions, extending it to environments with articulated robots and moving objects has proven difficult because of complex contact interactions and sudden pose changes [2]. The authors introduce a Graph-Structured Disentangled Representation that organizes the robot, objects, and background into independently optimizable Gaussian subfields within a scene graph [2]. A Task-Oriented Spatio-Temporal Alignment module stabilizes the process by exploiting the alternating Motion and Skill phases inherent in manipulation tasks to generate accurate pseudo-ground-truth trajectories [2]. A joint photometric-geometric optimization step then enforces temporal coherence and physical consistency, making the resulting scenes ready for simulation [2]. The framework was submitted to arXiv, an open-access repository that hosts electronic preprints across fields including computer science and mathematics [6]. arXiv, which began in 1991, passed the two-million-article milestone by the end of 2021 and receives roughly 24,000 submissions per month as of late 2024 [6]. Papers on the platform are moderated but not peer-reviewed [6]. The ManiSplat submission appears under the Computer Vision and Pattern Recognition category and is accessible through standard arXiv abstract pages, which also feature community-developed tools such as the Bibliographic Explorer and CORE Recommender via the arXivLabs framework [4][5]. The authors report that extensive experiments show the approach reconstructs interaction-driven dynamic scenes with high fidelity and controllability, supporting downstream robotic tasks and policy learning [2].