IGS’23

Abstract

Recent years have seen strong advances in learning general priors for 3D scene understanding tasks, while the burgeoning rise of interest in neural scene representations has reinvigorated per-scene optimization paradigms. In this talk, we discuss generative 3D tasks such as 3D reconstruction from the perspective of traditional optimization techniques and the incorporation of learned priors, along with how to effectively combine to two together. We start with 3D scene reconstruction optimization, followed by leveraging learned priors to infer unobserved information, performing geometric completion in 3D scans. We then explore learning a 3D texture manifold as a prior for test-time optimization to support texturing from arbitrary RGB image queries, where the learned prior serves as regularization across incongruous geometry and pose. Next, we introduce a new paradigm to learn a shape manifold from optimized neural fields that enables a dimension-agnostic approach for high-dimensional generative modeling. Finally, we explore how to learn 3D priors for scenes and how to enable new outlooks on 3D scene generative modeling.