Abstract:: Compared with traditional mechanical multi-line Lidar, area array Lidar can achieve greater field of view coverage through non-repetitive scanning, and has many applications in industrial production and robotics. A single sensor often has some limitations, while the fusion of multi-sensor data has higher precision and accuracy. Point cloud data contains accurate depth information of the environment, while image data contains rich color and texture information of the environment. The fusion of Point cloud and image can increase the dimensionality of the sensor's perception of environmental, and also enable robots and automation equipment to achieve more complex tasks and reconstruct colorful three-dimensional environments. The prerequisite for achieving the fusion of point cloud data and image data is to calibrate the extrinsic parameters of the Lidar and camera. The purpose of sensors external parameter calibration is to find the accurate position conversion relationship between the two sensor coordinate systems. In order to improve the calibration accuracy of array Lidar and monocular camera, this paper proposes a calibration method based on tetrahedral structure for array Lidar and camera. The tetrahedron structure formed by two isosceles right triangle calibration plates and the ground is used as the calibration object. Using the random sampling consensus algorithm to extract three plane features from point cloud data, The straight line parameter equations of the edges of the tetrahedral structure are obtained through the intersection of the three planes. The common points of the three planes are the vertices of the tetrahedral structure. The line segment detection algorithm is used to extract the straight line features of the intersection of three planes in the image data, and the vertex coordinates of the tetrahedral structure are obtained through the straight line intersection. The method of indirectly obtaining point and linear features through plane extraction is more accurate than the direct extraction method. The back-projection rays of the corner points in the image and the straight-line features in the point cloud form multiple sets of skew lines. The distance between the skew lines is used to construct the residual equation of the rotation and translation relationship. The angle error between the projection of the straight line feature in the point cloud on the image and the real imaging straight line is used to establish the residual equation of the rotation relationship. The "point-three-line" correspondence relationship between point cloud data and image data is established, which improves the constraint strength between corresponding features in point cloud and image data. Based on the Rodriguez formula, a rotation vector is used to represent the transformation matrix between the Lidar and camera coordinate systems. The transformation vector between the camera and radar coordinate systems is solved through nonlinear optimization and the transformation matrix is finally obtained. The distance from the projected point of the edge in the point cloud to the edge in the image is used as the projection error from the point cloud data to the image data. The proposed method is experimentally compared with the Livox company' s open-source calibration method and the University of Hong Kong's targetless calibration method. The tetrahedral structure data collected in this paper and the open-source algorithm data were tested, respectively. The calibration results on images with a resolution of 1 920x1 080 shows that the average projection error of the proposed method is within 0.6 pixels, which is superior to the other two calibration methods. At the same time, without a specific tetrahedral structure calibration object. The proposed method can use the wall corner tetrahedral structure to complete the calibration, which reflects the flexibility of the proposed method. The proposed high-precision calibration method of Lidar and camera provides a research foundation for the subsequent research on the fusion of Lidar data and image data.

A Combined Sensor System of Digital Camera with Lidar

Automatic Calibration Between Multi-Lines LiDAR and Visible Light Camera Based on Edge Refinement and Virtual Mask Matching

A fusion imaging system with visible light camera and photon counting lidar

Single-photon Computational 3D Imaging at 45 Km

Design of Lidar Receiving Optical System with Large FoV and High Concentration of Light to Resist Background Light Interference

Scanning single-pixel imaging lidar

3D Reconstruction & Assessment Framework based on affordable 2D Lidar

Multi-beam single-photon-counting three-dimensional imaging lidar

High-resolution Single-Photon LiDAR Without Range Ambiguity Using Hybrid-Mode Imaging [Invited]

Research on a Data Fusion Scheme for a Depth Camera and LIDAR of Mobile Robot Based on Depth Completion

A Robust 2D Lidar SLAM Method in Complex Environment

Analysis of Scannerless Imaging Lidar in Large Field of View Condition

Development of digital surface model and feature extraction by integrating laser scanner and ccd sensor with imu

A Multi-Lidar-based Point Cloud Acquisition Platform and Data Fusion for Autonomous Vehicle in Complex Urban Environment

LI-SLAM: Fusing LiDAR and Infrared Camera for Simultaneous Localization and Mapping

Laser Scanning Measurement based on Event Cameras.

Compact infrared imaging lidar system for obstacle detection and mapping in sensor fusion

A Multi-Sensor Fusion Framework for Localization Using LiDAR, IMU and RGB-DCamera

Planar Array Lidar and Camera Calibration Method Based on Tetrahedral Features

Hybrid Lidar System Using Combination Of Time Correlated Single Photon Counting And Waveform Sampling

Integration of Vision and Laser Dis Sensor for Efficient and Precise