Thinking in Boxes: 3D Editing in Real Images Made Easy

A new computer vision method lets users edit real photographs by placing and moving 3D boxes, giving precise control over object translation, rotation, scaling, and viewpoint changes that earlier text-based interfaces could not achieve [1]. The system, described in a paper submitted to arXiv on 18 June 2026, treats image editing as a geometry problem. A user specifies an input box and an output box around an object, and the model executes the corresponding spatial transformation while preserving the object’s identity and recovering previously unseen regions [1][2]. Each face of the box is color-coded to convey 3D orientation, an approach the authors call “thinking in boxes” [2]. Prior editing tools relied on text prompts or 2D conditioning, which provided only weak, ambiguous control — particularly when an edit required large object motions or camera-angle shifts [2]. Earlier work that did incorporate 3D primitives used boxes merely as loose signals for approximate object location, not as precise transformation specifications [2]. To keep generated results consistent with the surrounding scene, the researchers introduce a depth-aligned planar floor that serves as a global reference frame. The floor is shaded with depth-aware cues, giving the image generator a stable spatial anchor even under large transformations [2]. The model is trained in two stages. The first stage uses synthetic multi-object scenes; the second stage fine-tunes on a small set of real-world videos drawn from the Objectron dataset [1][2]. Despite the limited real-world training data, the system generalizes to complex, in-the-wild photographs and substantially outperforms recent state-of-the-art methods on large 3D edits [1][2]. The work was posted on arXiv under the Computer Vision and Pattern Recognition category and is available in both PDF and HTML formats. The paper’s page also links to code and data repositories through services such as CatalyzeX, Hugging Face, and DagsHub, though the repositories themselves were not reviewed for this report [1]. The project is associated with arXivLabs, a framework that lets collaborators develop and share experimental features directly on the arXiv website under a set of community values that include openness and user-data privacy [1].