Overview
Recent 3D reconstruction methods (most notably, NeRF [1] and Gaussian Splatting [2]) learn a 3D representation of the scene from a set of 2D images. While these representations are able to model the scene with a very high visual accuracy, they make edition cumbersome and they are unable to infer parts of the objects which are unseen in the initial images. In particular, 3D object extraction remains a challenge and would allow the use of the reconstructed objects in video games or in CGI.
Objectives
-
Extract meaningful 3D objects from the reconstruction, with or without user interaction
-
Leverage a prior from a diffusion model to infer parts of the objects which are unseen
Prerequisites
-
Python proficiency, familiarity with Pytorch
-
Experience with running large projects in bash
-
Understanding of 3D camera model
-
Knowledge of modern 3D reconstruction techniques [1,2] is a plus
Contact
This project will be conducted in collaboration with CVLab. Contact [email protected] for more information.
References
[1] NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis, ECCV 2020
[2] 3D Gaussian Splatting for Real-Time Radiance Field Rendering, SIGGRAPH 2023