Abstract:Today, three-dimensional reconstruction of objects has many applications in various fields, and therefore, choosing a suitable method for high resolution three-dimensional reconstruction is an important issue and displaying high-level details in three-dimensional models is a serious challenge in this field. Until now, active methods have been used for high-resolution three-dimensional reconstruction. But the problem of active three-dimensional reconstruction methods is that they require a light source close to the object. Shape from polarization (SfP) is one of the best solutions for high-resolution three-dimensional reconstruction of objects, which is a passive method and does not have the drawbacks of active methods. The changes in polarization of the reflected light from an object can be analyzed by using a polarization camera or locating polarizing filter in front of the digital camera and rotating the filter. Using this information, the surface normal can be reconstructed with high accuracy, which will lead to local reconstruction of the surface details. In this paper, an end-to-end deep learning approach has been presented to produce the surface normal of objects. In this method a benchmark dataset has been used to train the neural network and evaluate the results. The results have been evaluated quantitatively and qualitatively by other methods and under different lighting conditions. The MAE value (Mean-Angular-Error) has been used for results evaluation. The evaluations showed that the proposed method could accurately reconstruct the surface normal of objects with the lowest MAE value which is equal to 18.06 degree on the whole dataset, in comparison to previous physics-based methods which are between 41.44 and 49.03 degree.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the accurate reconstruction of surface normals in high - resolution three - dimensional reconstruction. Specifically, the paper proposes a deep - learning method based on polarized light, aiming to accurately estimate the surface normals of objects by using only polarized images, thereby achieving high - resolution three - dimensional reconstruction. ### Problem Background Traditional three - dimensional reconstruction methods can be divided into active methods and passive methods: - **Active methods**: Rely on external light sources (such as lasers, structured light, etc.). These methods have good effects in high - resolution reconstruction, but require the light source to be close to the object, which limits the application scenarios. - **Passive methods**: Do not interfere with the object being reconstructed, and only use sensors sensitive to visible light to measure the reflected or emitted radiation, and infer the three - dimensional structure of the object through image analysis. Shape from Polarization (SfP) in passive methods is a very promising method. It uses the polarization characteristics of light to obtain the surface information of the object, and has the advantages of non - contact and non - interference. ### Core Problems of the Paper The paper points out that although the SfP method has its advantages, it still has some challenges, such as phase ambiguity and other problems. In order to solve these problems and improve the reconstruction accuracy of surface normals, the author proposes a method based on a convolutional neural network (CNN), especially the U - Net architecture, and combines ResNet18 as the backbone network to directly estimate the surface normals from polarized images. ### Specific Objectives 1. **Solve the phase ambiguity problem**: By using a deep - learning model, avoid the phase ambiguity problem in traditional physical methods. 2. **Improve reconstruction accuracy**: Under different lighting conditions, achieve higher surface normal reconstruction accuracy than existing physical methods. 3. **Simplify data requirements**: Only use polarized images without additional physical parameters or information. ### Experimental Results The experimental results show that this method performs excellently on multiple test objects, with an average angular error (MAE) of only 18.06 degrees, which is significantly better than other physical - based methods (MAE values are between 41.44 and 49.03 degrees). In addition, this method shows stable performance under different lighting conditions (indoor, sunny outdoor, cloudy outdoor). In conclusion, the main contribution of this paper is to propose an innovative deep - learning framework that can efficiently and accurately reconstruct the surface normals of objects from polarized images, thereby providing a new solution for high - resolution three - dimensional reconstruction.

Surface Normal Reconstruction Using Polarization-Unet

High Resolution Surface Reconstruction of Cultural Heritage Objects Using Shape from Polarization Method

On accurate recovery of 3D surface normal using minimum 2D images

Shape from Polarization with Distant Lighting Estimation.

A Novel Learning Based Non-Lambertian Photometric Stereo Method for Pixel-Level Normal Reconstruction of Polished Surfaces

3D Human Shape Reconstruction from a Polarization Image

Near-infrared monocular 3D computational polarization imaging of surfaces exhibiting nonuniform reflectance

GNeRP: Gaussian-guided Neural Reconstruction of Reflective Objects with Noisy Polarization Priors

PISR: Polarimetric Neural Implicit Surface Reconstruction for Textureless and Specular Objects

GR-PSN: Learning to Estimate Surface Normal and Reconstruct Photometric Stereo Images.

Estimating High-resolution Surface Normals via Low-resolution Photometric Stereo Images

Shape from Polarization of low-texture objects with rough depth information

NeRSP: Neural 3D Reconstruction for Reflective Objects with Sparse Polarized Images

SS-SfP:Neural Inverse Rendering for Self Supervised Shape from (Mixed) Polarization

Human Pose and Shape Estimation from Single Polarization Images

Depth Sensing Using Geometrically Constrained Polarization Normals

Deep Photometric Stereo Networks for Determining Surface Normal and Reflectances.

High-Fidelity Mask-free Neural Surface Reconstruction for Virtual Reality

Polarimetric Multi-View Inverse Rendering

End-to-end neural network for pBRDF estimation of object to reconstruct polarimetric reflectance

Polarimetric Multi-View 3d Reconstruction Based On Parallax Angle And Zenith Angle Optimization