I2-SDF: Intrinsic Indoor Scene Reconstruction and Editing Via Raytracing in Neural SDFs
Jingsen Zhu,Yuchi Huo,Qi Ye,Fujun Luan,Jifan Li,Dianbing Xi,Lisha Wang,Rui Tang,Wei Hua,Hujun Bao,Rui Wang
DOI: https://doi.org/10.1109/cvpr52729.2023.01202
2023-01-01
Abstract:In this work, we present I 2 -SDF, a new method for intrinsic indoor scene reconstruction and editing using differentiable Monte Carlo raytracing on neural signed distance fields (SDFs). Our holistic neural SDF-based frame-work jointly recovers the underlying shapes, incident radiance and materials from multi-view images. We introduce a novel bubble loss for fine-grained small objects and error-guided adaptive sampling scheme to largely improve the reconstruction quality on large-scale indoor scenes. Further, we propose to decompose the neural radiance field into spatially-varying material of the scene as a neural field through surface-based, differentiable Monte Carlo raytracing and emitter semantic segmentations, which enables physically based and photorealistic scene relighting and editing applications. Through a number of qualitative and quantitative experiments, we demonstrate the superior quality of our method on indoor scene reconstruction, novel view synthesis, and scene editing compared to state-of-the-art baselines. Our project page is at https://jingsenzhu.github.io/i2-sdf.