Generative Diffusion Priors for 3D Mapping of the Dark Universe

Researchers have proposed two new methods for advancing 3D mapping and keypoint estimation, one for reconstructing dark matter distribution and another for predicting 3D keypoints from a single image.

A team of researchers has introduced a novel approach for 3D mapping of the dark universe using generative diffusion priors and high-resolution cosmological simulations. This method reconstructs the three-dimensional distribution of dark matter from weak-lensing observations and leverages a new dataset called Conicus3D to learn a data-driven diffusion-model prior^[1]. Meanwhile, another research team has developed a framework called KeyDiff3D, which can accurately predict 3D keypoints from a single image without manual annotations or calibrated multi-view images. KeyDiff3D leverages geometric priors from a pretrained multi-view diffusion model, eliminating the need for expensive data acquisitions^[2]. The dark matter mapping approach combines a learned prior with a differentiable physical forward model, yielding substantially improved 2D and 3D reconstruction accuracy over baseline methods. The KeyDiff3D framework also enables manipulation of 3D objects generated by the diffusion model.