Abstract:This paper presents a novel method for fully automatic and convenient extrinsic calibration of a 3D LiDAR and a panoramic camera with a normally printed chessboard. The proposed method is based on the 3D corner estimation of the chessboard from the sparse point cloud generated by one frame scan of the LiDAR. To estimate the corners, we formulate a full-scale model of the chessboard and fit it to the segmented 3D points of the chessboard. The model is fitted by optimizing the cost function under constraints of correlation between the reflectance intensity of laser and the color of the chessboard's patterns. Powell's method is introduced for resolving the discontinuity problem in optimization. The corners of the fitted model are considered as the 3D corners of the chessboard. Once the corners of the chessboard in the 3D point cloud are estimated, the extrinsic calibration of the two sensors is converted to a 3D-2D matching problem. The corresponding 3D-2D points are used to calculate the absolute pose of the two sensors with Unified Perspective-n-Point (UPnP). Further, the calculated parameters are regarded as initial values and are refined using the Levenberg-Marquardt method. The performance of the proposed corner detection method from the 3D point cloud is evaluated using simulations. The results of experiments, conducted on a Velodyne HDL-32e LiDAR and a Ladybug3 camera under the proposed re-projection error metric, qualitatively and quantitatively demonstrate the accuracy and stability of the final extrinsic calibration parameters.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is how to achieve fully - automatic, convenient and accurate extrinsic calibration of 3D LiDAR and panoramic cameras. Specifically, the author proposes a new method to complete this task using a commonly printed checkerboard pattern.
### Main problems
1. **Multi - modal data fusion**: Combine the data of LiDAR (providing depth information) and panoramic cameras (providing color and texture information) to achieve more abundant perception capabilities, such as 3D mapping, model generation and pedestrian detection, etc.
2. **Challenges in extrinsic calibration**: In order to effectively fuse the data from LiDAR and cameras, the relative position and pose (i.e., extrinsic parameters) of the two must be accurately calculated. Traditional extrinsic calibration methods either require human intervention or rely on complex geometric feature extraction, and these methods have inconvenience or insufficient precision in practical applications.
### Paper solutions
The author proposes a method based on checkerboard corner point estimation. It uses the reflection intensity information in the LiDAR point cloud to automatically identify the corners of the checkerboard and match them with the corners in the image, thereby achieving extrinsic calibration. The specific steps are as follows:
1. **Checkerboard detection and segmentation**:
- Automatically detect and segment the checkerboard area from the LiDAR point cloud.
- Use the planarity and boundary characteristics of the point cloud to screen out checkerboard candidate segments.
2. **Corner point estimation**:
- Construct a full - scale model of the checkerboard and fit the model to the segmented 3D point cloud by optimizing the cost function.
- Use the Powell method to solve the discontinuity problem in the optimization process to ensure the accuracy of corner point estimation.
3. **3D - 2D matching**:
- Establish a correspondence between the estimated 3D corner points and the 2D corner points in the image, transforming it into a 3D - 2D matching problem.
- Use the Unified Perspective - n - Point (UPnP) method to calculate the initial extrinsic parameter matrix.
4. **Parameter optimization**:
- Use the Levenberg - Marquardt method to perform nonlinear optimization on the initial extrinsic parameters to further improve the calibration accuracy.
### Contributions
1. **Innovation**: For the first time, use the reflection intensity information to estimate the checkerboard corner points from the point cloud, which improves the accuracy and robustness of corner point detection.
2. **Automation**: The entire process does not require human intervention, achieving full - process automation from checkerboard detection, corner point estimation to extrinsic parameter optimization.
3. **Evaluation and implementation**: Verify the effectiveness of the method through simulation and experiments, and provide Python implementation code for subsequent research and application.
### Formula display
- **Theoretical maximum number of points**:
\[
n_{\text{theo}} \approx \left\lfloor \frac{d_W}{2r \sin\left(\frac{\Delta h}{2}\right)} \right\rfloor \left\lfloor \frac{d_H}{2r \sin\left(\frac{\Delta v}{2}\right)} \right\rfloor
\]
where \( d_W \) and \( d_H \) are the width and height of the checkerboard respectively, \( r \) is the distance from the center of the checkerboard to LiDAR, and \( \Delta h \) and \( \Delta v \) are the horizontal and vertical angular resolutions respectively.
- **Definition of gray - level interval**:
\[
\begin{cases}
\tau_l = \frac{(\epsilon_g - 1)R_L + R_H}{\epsilon_g} \\
\tau_h = \frac{R_L + (\epsilon_g - 1)R_H}{\epsilon_g}
\end{cases}
\]
where \( R_L \) and \( R_H \) are the reflection intensity values corresponding to the low - intensity and high - intensity peaks respectively, and \( \epsilon_g \geq 2 \) is a constant.
Through the above methods, this paper effectively solves the problem of extrinsic calibration of 3D LiDAR and panoramic cameras, providing for multi - modal data fusion.