Abstract:Integrating multiple types of sensors into autonomous systems, such as cars and robots, has become a widely adopted approach in modern technology. Among these sensors, RGB cameras, thermal cameras, and LiDAR are particularly valued for their ability to provide comprehensive environmental data. However, despite their advantages, current research primarily focuses on the one or combination of two sensors at a time. The full potential of utilizing all three sensors is often neglected. One key challenge is the ego-motion compensation of data in dynamic situations, which results from the rotational nature of the LiDAR sensor, and the blind spots of standard cameras due to their limited field of view. To resolve this problem, this paper proposes a novel method for the simultaneous registration of LiDAR, panoramic RGB cameras, and panoramic thermal cameras in dynamic environments without the need for calibration targets. Initially, essential features from RGB images, thermal data, and LiDAR point clouds are extracted through a novel method, designed to capture significant raw data characteristics. These extracted features then serve as a foundation for ego-motion compensation, optimizing the initial dataset. Subsequently, the raw features can be further refined to enhance calibration accuracy, achieving more precise alignment results. The results of the paper demonstrate the effectiveness of this approach in enhancing multiple sensor calibration compared to other ways. In the case of a high speed of around 9 m/s, some situations can improve the accuracy about 30 percent higher for LiDAR and Camera calibration. The proposed method has the potential to significantly improve the reliability and accuracy of autonomous systems in real-world scenarios, particularly under challenging environmental conditions.

Single-Shot is Enough: Panoramic Infrastructure Based Calibration of Multiple Cameras and 3D LiDARs

LiDAR-Camera Calibration under Arbitrary Configurations: Observability and Methods

Spatiotemporal Decoupling Based LiDAR-Camera Calibration under Arbitrary Configurations.

Intrinsic and Extrinsic Calibration of Roadside LiDAR and Camera

A Long-focus Camera and Telemetry LiDAR Calibration for Rapid Transit Equipment.

Extrinsic Calibration of A 3d Lidar and A Camera Using A Trihedron

Observability-Aware Intrinsic and Extrinsic Calibration of LiDAR-IMU Systems

A Target-based Calibration Method for LiDAR-Visual-Thermal Multi-Sensor System.

Automated Extrinsic Calibration of Multi-cameras and LiDAR

Automatic Extrinsic Calibration of 3D LIDAR and Multi-Cameras Based on Graph Optimization

Automatic Calibration Between Multi-Lines LiDAR and Visible Light Camera Based on Edge Refinement and Virtual Mask Matching

Automatic extrinsic calibration for structured light camera and repetitive LiDARs

Automatic Multi-camera Multi-LiDAR Extrinsic Calibration Using Geometric Corners.

Calibration of a Single-Point Laser Range Finder and a Camera

LiDAR-360 RGB Camera-360 Thermal Camera Targetless Calibration for Dynamic Situations

Multi-Dimensional Geometric Feature-Based Calibration Method for LiDAR and Camera Fusion.

LCE-Calib: Automatic LiDAR-Frame/Event Camera Extrinsic Calibration With A Globally Optimal Solution

Multipath-Closure Calibration of Stereo Camera and 3D LiDAR Combined with Multiple Constraints

Robust Plane-Based Calibration of Multiple Non-Overlapping Cameras

Extrinsic Calibration of Camera and LiDAR Systems With Three‐Dimensional Towered Checkerboards

SceneCalib: Automatic Targetless Calibration of Cameras and Lidars in Autonomous Driving