Abstract:: Compared with traditional mechanical multi-line Lidar, area array Lidar can achieve greater field of view coverage through non-repetitive scanning, and has many applications in industrial production and robotics. A single sensor often has some limitations, while the fusion of multi-sensor data has higher precision and accuracy. Point cloud data contains accurate depth information of the environment, while image data contains rich color and texture information of the environment. The fusion of Point cloud and image can increase the dimensionality of the sensor's perception of environmental, and also enable robots and automation equipment to achieve more complex tasks and reconstruct colorful three-dimensional environments. The prerequisite for achieving the fusion of point cloud data and image data is to calibrate the extrinsic parameters of the Lidar and camera. The purpose of sensors external parameter calibration is to find the accurate position conversion relationship between the two sensor coordinate systems. In order to improve the calibration accuracy of array Lidar and monocular camera, this paper proposes a calibration method based on tetrahedral structure for array Lidar and camera. The tetrahedron structure formed by two isosceles right triangle calibration plates and the ground is used as the calibration object. Using the random sampling consensus algorithm to extract three plane features from point cloud data, The straight line parameter equations of the edges of the tetrahedral structure are obtained through the intersection of the three planes. The common points of the three planes are the vertices of the tetrahedral structure. The line segment detection algorithm is used to extract the straight line features of the intersection of three planes in the image data, and the vertex coordinates of the tetrahedral structure are obtained through the straight line intersection. The method of indirectly obtaining point and linear features through plane extraction is more accurate than the direct extraction method. The back-projection rays of the corner points in the image and the straight-line features in the point cloud form multiple sets of skew lines. The distance between the skew lines is used to construct the residual equation of the rotation and translation relationship. The angle error between the projection of the straight line feature in the point cloud on the image and the real imaging straight line is used to establish the residual equation of the rotation relationship. The "point-three-line" correspondence relationship between point cloud data and image data is established, which improves the constraint strength between corresponding features in point cloud and image data. Based on the Rodriguez formula, a rotation vector is used to represent the transformation matrix between the Lidar and camera coordinate systems. The transformation vector between the camera and radar coordinate systems is solved through nonlinear optimization and the transformation matrix is finally obtained. The distance from the projected point of the edge in the point cloud to the edge in the image is used as the projection error from the point cloud data to the image data. The proposed method is experimentally compared with the Livox company' s open-source calibration method and the University of Hong Kong's targetless calibration method. The tetrahedral structure data collected in this paper and the open-source algorithm data were tested, respectively. The calibration results on images with a resolution of 1 920x1 080 shows that the average projection error of the proposed method is within 0.6 pixels, which is superior to the other two calibration methods. At the same time, without a specific tetrahedral structure calibration object. The proposed method can use the wall corner tetrahedral structure to complete the calibration, which reflects the flexibility of the proposed method. The proposed high-precision calibration method of Lidar and camera provides a research foundation for the subsequent research on the fusion of Lidar data and image data.

A simple and effective extrinsic calibration method of a camera and a single line scanning lidar

Extrinsic Calibration of a Single-Point Laser Rangefinder and Single Camera

Extrinsic Calibration of A 3d Lidar and A Camera Using A Trihedron

Intrinsic and Extrinsic Calibration of Roadside LiDAR and Camera

LiDAR-Camera Calibration under Arbitrary Configurations: Observability and Methods

3d Lidar-Camera Extrinsic Calibration Using An Arbitrary Trihedron

A Long-focus Camera and Telemetry LiDAR Calibration for Rapid Transit Equipment.

A Direct Method for the Extrinsic Calibration of a Camera and a Line Scan LIDAR

A New Extrinsic Calibration Method of Three-Dimensional Imaging Lidar and Camera

IMU-Assisted Target-Free Extrinsic Calibration of Heterogeneous Lidars Based on Continuous-Time Optimization

A Method of Extrinsic Calibration Between a Four-Layer Laser Range Finder and a Camera

Extrinsic Calibration between a Camera and a 2D Laser Rangefinder using a Photogrammetric Control Field

The Extrinsic Calibration of Area-Scan Camera and 2D Laser Rangefinder (LRF) Using Checkerboard Trihedron

Extrinsic Calibration for Multi-Lidar Systems Involving Heterogeneous Laser Scanning Models.

Targetless Extrinsic Calibration for Multi-beam LIDARs with Narrow Overlapping Field of View

A New Algorithm for the Extrinsic Calibration of a 2D LIDAR and a Camera

LiDAR-camera system extrinsic calibration by establishing virtual point correspondences from pseudo calibration objects.

Targetless Extrinsic Calibration of Camera and Low-resolution 3D LiDAR

Planar Array Lidar and Camera Calibration Method Based on Tetrahedral Features

Accurate and Automatic Extrinsic Calibration for a Monocular Camera and Heterogenous 3D LiDARs

CPI: LiDAR-Camera Extrinsic Calibration Based on Feature Points with Reflection Intensity