Abstract:LiDAR-camera fusion is one of the core processes for the perception system of current automated driving systems. The typical sensor fusion process includes a list of coordinate transformation operations following system calibration. Although a significant amount of research has been done to improve the fusion accuracy, there are still inherent data mapping errors in practice related to system synchronization offsets, vehicle vibrations, the small size of the target, and fast relative moving speeds. Moreover, more and more complicated algorithms to improve fusion accuracy can overwhelm the onboard computational resources, limiting the actual implementation. This study proposes a novel and low-cost probabilistic LiDAR-Camera fusion method to alleviate these inherent mapping errors in scene reconstruction. By calculating shape similarity using KL-divergence and applying RANSAC-regression-based trajectory smoother, the effects of LiDAR-camera mapping errors are minimized in object localization and distance estimation. Designed experiments are conducted to prove the robustness and effectiveness of the proposed strategy.
What problem does this paper attempt to address?
### Problems the paper attempts to solve
This paper aims to solve the mapping error problems encountered in LiDAR - camera fusion in autonomous vehicles. Although a great deal of research has been dedicated to improving the fusion accuracy, in practical applications, there are still inherent data mapping errors due to factors such as system synchronization offsets, vehicle vibrations, small target object sizes, and high relative motion speeds. Moreover, although complex algorithms can improve the fusion accuracy, they will occupy a large amount of on - board computing resources, which limits their practical applications. Therefore, this paper proposes a new and low - cost probabilistic LiDAR - camera fusion method to mitigate the impact of these inherent mapping errors on scene reconstruction.
### Specific problem descriptions
1. **Mapping of background noise to the region of interest**: In practical applications, background points may be wrongly mapped to the regions of interest (AOIs) and become noise.
2. **Offset of projection points**: The projection of LiDAR points on the image may be offset, resulting in the inability to correctly match the corresponding pixel points.
### Solutions
The paper proposes a multi - step probabilistic method to deal with these inevitable mapping errors:
1. **Enlarging AOIs**: First, map larger AOIs onto the image to deal with the problem of projection point offset.
2. **Probabilistic filtering**: Then, apply a probabilistic filtering process to remove noise and select the correct mapping results.
### Experimental verification
To verify the effectiveness and robustness of the proposed method, a series of experiments were designed, especially tested in the scenario of car - electric scooter interactions. The electric scooter was chosen as the experimental object because this micro - mobility tool is one of the most challenging targets for the autonomous driving perception system, with its small size and high moving speed.
### Main contributions
1. **Proposing a method to deal with LiDAR - camera mapping errors in practical applications**: It can effectively improve the accuracy of fusion results even when mapping errors cannot be avoided.
2. **Low computing resource requirements**: This method requires fewer computing resources and can complement the existing efforts to improve mapping accuracy.
3. **Shape descriptor based on KL divergence**: It can efficiently capture geometric information from LiDAR point distributions and simplify similarity calculations.
4. **Support for micro - mobility vehicle detection and localization**: This method can effectively support the detection and localization of micro - mobility vehicles such as electric scooters.
### Mathematical formulas
1. **Mapping relationship**:
\[
M(x, y, z)=(l, w)+(\epsilon_l, \epsilon_w)
\]
where \((x, y, z)\) are the LiDAR point coordinates, \((l, w)\) are the image pixel coordinates, and \((\epsilon_l, \epsilon_w)\) are the mapping errors.
2. **Mode clustering**:
\[
\text{Mode}(M^{-1}(c_k^*))=(m_1, m_2, \ldots, m_n)
\]
where \(c_k^*\) is the bounding box of the target object, and \(m_1, m_2, \ldots, m_n\) are the candidate LiDAR point clusters.
3. **KL - divergence calculation**:
\[
D_{KL}(P \| Q)=\sum_{x \in X} P(x) \log \left(\frac{P(x)}{Q(x)}\right)
\]
where \(P\) and \(Q\) are two discrete probability distributions, representing the rotated shape descriptor vectors.
4. **Selecting the best cluster**:
\[
m^*=\arg\max_{m_1, m_2, \ldots, m_n}\left(D_{KL}(\text{Shape}(m_1) \| S^*), D_{KL}(\text{Shape}(m_2) \| S^*), \ldots, D_{KL}(\text{Shape}(m_n) \| S^*)\right)
\]
where \(\text{Shape}(.)\)