Enhancing PSNeRF with Shape-from-Silhouette for efficient and accurate 3D reconstruction

Lyes Abada,Aimen Said Mezabiat,Tarek Gacem,Omar Chouaab Malki,Mohamed Mekkaoui
DOI: https://doi.org/10.1007/s11042-024-20319-3
IF: 2.577
2024-10-08
Multimedia Tools and Applications
Abstract:The three-dimensional reconstruction of objects from images is a field of computer vision. Its fundamental aim is to determine the geometry of an object from visual data. Among the techniques in this field, we can find multi-view photometric stereo, which uses several images taken from different camera positions under different lighting conditions. This paper introduces a novel enhancement to the PSNeRF method by incorporating initial object generation through the Shape from Silhouette (SFS) technique. By training both color and occupancy simultaneously, we ensure compatibility between the initial SFS object and the PSNeRF model. Our approach addresses the challenges of high processing time and slow convergence in PSNeRF, leading to improved accuracy and efficiency in 3D reconstruction. To address this issue, we leveraged the Shape from Silhouette (SFS) technique, a rapid and efficient method for generating an initial object. We adapted a deep learning model inspired by the NeRF technique, training it on SFS data to quickly obtain an acceptable initial approximation of the object. In the experimental section of this paper, we present the results obtained, highlighting the performance of our approach and discussing the implications of our contributions. We demonstrate that the proposed method significantly outperforms the basic PSNeRF technique in terms of both quality and processing time.
computer science, information systems, theory & methods,engineering, electrical & electronic, software engineering
What problem does this paper attempt to address?