Abstract:Lidar has become a cornerstone sensing modality for 3D vision, especially for large outdoor scenarios and autonomous driving. Conventional lidar sensors are capable of providing centimeter-accurate distance information by emitting laser pulses into a scene and measuring the time-of-flight (ToF) of the reflection. However, the polarization of the received light that depends on the surface orientation and material properties is usually not considered. As such, the polarization modality has the potential to improve scene reconstruction beyond distance measurements. In this work, we introduce a novel long-range polarization wavefront lidar sensor (PolLidar) that modulates the polarization of the emitted and received light. Departing from conventional lidar sensors, PolLidar allows access to the raw time-resolved polarimetric wavefronts. We leverage polarimetric wavefronts to estimate normals, distance, and material properties in outdoor scenarios with a novel learned reconstruction method. To train and evaluate the method, we introduce a simulated and real-world long-range dataset with paired raw lidar data, ground truth distance, and normal maps. We find that the proposed method improves normal and distance reconstruction by 53\% mean angular error and 41\% mean absolute error compared to existing shape-from-polarization (SfP) and ToF methods. Code and data are open-sourced at <a class="link-external link-https" href="https://light.princeton.edu/pollidar" rel="external noopener nofollow">this https URL</a>.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: How to improve the accuracy of large - scene reconstruction by combining polarization analysis and LiDAR (Light Detection and Ranging) sensors, especially in applications that require high - precision 3D vision such as autonomous driving. Traditional LiDAR sensors mainly rely on time - of - flight (ToF) to measure distance information while ignoring the polarization characteristics of light, which can provide additional information about surface normals and material properties.
### Specific description of the problem
1. **Limitations of traditional LiDAR**:
- Traditional LiDAR sensors can only provide centimeter - level distance information and cannot effectively use polarization characteristics to enhance scene reconstruction.
- Polarization characteristics are closely related to surface normals and material properties, but existing sensors usually only detect intensity and ignore polarization information.
2. **Potential of polarization characteristics**:
- Polarization characteristics can provide information about surface normals and material properties, which is helpful for improving the reconstruction accuracy of distance and normals.
- In outdoor scenes, especially in autonomous driving scenes, polarization characteristics can significantly improve the reconstruction quality in the long - distance range.
### The method proposed in the paper
To solve the above problems, the paper introduces a new type of polarization - wavefront LiDAR sensor (PolLidar), which can modulate the polarization state of the transmitted and received light and directly read the time - resolved polarization - wavefront signal. Specifically:
- **Sensor design**: PolLidar obtains the original time - resolved polarization - wavefront data by modulating the polarization state of the transmitted and received light.
- **Reconstruction method**: Use a neural network to estimate surface normals, distances, and material properties from the original polarization - wavefront data.
- **Dataset**: To train and evaluate the model, the paper introduces a long - distance dataset containing simulated and real - world data, providing paired original LiDAR data, ground - truth distances, and normal maps.
### Experimental results
The experimental results show that compared with the existing shape - from - polarization (SfP) and ToF methods, the proposed method reduces the average angular error by 53% in normal estimation and the average absolute error by 41% in distance estimation.
### Formula representation
The formulas involved in the paper include:
1. **Reflectance model**:
\[
M(\tau, \omega_i, \omega_o) = M_s(\tau, \omega_i, \omega_o)+M_d(\tau, \omega_i, \omega_o)
\]
where \(M_s\) and \(M_d\) represent the specular reflection and diffuse reflection parts respectively.
2. **LiDAR forward model**:
\[
I(t, \omega)=\left[\int_{0}^{t'} H(\tau, \omega, \omega)s_{\text{laser}}(\omega, t'-\tau)d\tau\right]_0
\]
where \(t' = t-\frac{2d}{c}\), \(d\) is the distance from the laser to the scene, and \(c\) is the speed of light.
3. **Loss function**:
- Normal loss:
\[
L_{\text{normal}}=\vert1-(c\odot n_{\text{gt}})\cdot(c\odot\hat{n})\vert_1
\]
- Distance loss:
\[
L_{\text{dist}}=\vert c\odot d_{\text{gt}}-c\odot\hat{d}\vert_1
\]
In summary, this paper aims to improve the 3D reconstruction accuracy in large - scale scenes, especially in autonomous driving scenes, by introducing a polarization - wavefront LiDAR sensor and the corresponding reconstruction method.