TensoSDF: Roughness-aware Tensorial Representation for Robust Geometry and Material Reconstruction

Jia Li,Lu Wang,Lei Zhang,Beibei Wang

2024-05-04

Abstract:Reconstructing objects with realistic materials from multi-view images is problematic, since it is highly ill-posed. Although the neural reconstruction approaches have exhibited impressive reconstruction ability, they are designed for objects with specific materials (e.g., diffuse or specular materials). To this end, we propose a novel framework for robust geometry and material reconstruction, where the geometry is expressed with the implicit signed distance field (SDF) encoded by a tensorial representation, namely TensoSDF. At the core of our method is the roughness-aware incorporation of the radiance and reflectance fields, which enables a robust reconstruction of objects with arbitrary reflective materials. Furthermore, the tensorial representation enhances geometry details in the reconstructed surface and reduces the training time. Finally, we estimate the materials using an explicit mesh for efficient intersection computation and an implicit SDF for accurate representation. Consequently, our method can achieve more robust geometry reconstruction, outperform the previous works in terms of relighting quality, and reduce 50% training times and 70% inference time.

Graphics

What problem does this paper attempt to address?

The problem this paper attempts to address is the reconstruction of objects with realistic materials from multi-view images. Due to the highly ill-posed nature of this task, existing methods struggle to decouple geometry and materials. Although neural rendering-based methods like NeRF have demonstrated significant reconstruction capabilities, they are typically designed for specific types of materials (such as diffuse or specular materials) and cannot handle objects with arbitrary reflective materials.

To overcome these limitations, the authors propose a new framework—TensoSDF, a roughness-aware tensor representation method for robustly reconstructing geometry and materials from multi-view images. Specifically:

1. **Fusion of Roughness-Aware Radiance and Reflectance Fields**: By combining the radiance field and the reflectance field, the model can robustly reconstruct the surfaces of objects with reflective properties and improve the quality of reconstruction details.

2. **Tensor Representation of SDF**: TensoSDF is introduced to represent the implicit signed distance field (SDF). This representation not only enhances the detail of the reconstructed surfaces but also reduces training time.

3. **Explicit and Implicit Fusion Strategy**: When estimating materials, the explicit mesh is combined with the implicit SDF. The explicit mesh improves the efficiency of ray intersection calculations, while the implicit SDF ensures the accuracy of geometric representation.

Through this method, the proposed model in the paper can achieve more robust geometric reconstruction and surpass previous works in relighting quality, while significantly reducing training and inference time.