Abstract:Integrating multiple types of sensors into autonomous systems, such as cars and robots, has become a widely adopted approach in modern technology. Among these sensors, RGB cameras, thermal cameras, and LiDAR are particularly valued for their ability to provide comprehensive environmental data. However, despite their advantages, current research primarily focuses on the one or combination of two sensors at a time. The full potential of utilizing all three sensors is often neglected. One key challenge is the ego-motion compensation of data in dynamic situations, which results from the rotational nature of the LiDAR sensor, and the blind spots of standard cameras due to their limited field of view. To resolve this problem, this paper proposes a novel method for the simultaneous registration of LiDAR, panoramic RGB cameras, and panoramic thermal cameras in dynamic environments without the need for calibration targets. Initially, essential features from RGB images, thermal data, and LiDAR point clouds are extracted through a novel method, designed to capture significant raw data characteristics. These extracted features then serve as a foundation for ego-motion compensation, optimizing the initial dataset. Subsequently, the raw features can be further refined to enhance calibration accuracy, achieving more precise alignment results. The results of the paper demonstrate the effectiveness of this approach in enhancing multiple sensor calibration compared to other ways. In the case of a high speed of around 9 m/s, some situations can improve the accuracy about 30 percent higher for LiDAR and Camera calibration. The proposed method has the potential to significantly improve the reliability and accuracy of autonomous systems in real-world scenarios, particularly under challenging environmental conditions.

Robust Intrinsic and Extrinsic Calibration of RGB-D Cameras

Extrinsic Calibration of A 3d Lidar and A Camera Using A Trihedron

Intrinsic and Extrinsic Calibration of Roadside LiDAR and Camera

Simultaneous Intrinsic And Extrinsic Calibration Of A Visual-Odometric Sensor System

Calibration of a Single-Point Laser Range Finder and a Camera

Accurate Intrinsic Calibration Of Depth Camera With Cuboids

A Linear Approach for Depth and Colour Camera Calibration Using Hybrid Parameters

Robust 3D Reconstruction with an RGB-D Camera

A Novel Method for Extrinsic Calibration of Multiple RGB-D Cameras Using Descriptor-Based Patterns

PeLiCal: Targetless Extrinsic Calibration via Penetrating Lines for RGB-D Cameras with Limited Co-visibility

Robust Self-Supervised Extrinsic Self-Calibration

A Robust Extrinsic Calibration Framework for Vehicles with Unscaled Sensors

Bi-objective Framework for Sensor Fusion in RGB-D Multi-View Systems: Applications in Calibration

3D Radar and Camera Co-Calibration: A Flexible and Accurate Method for Target-based Extrinsic Calibration

LCE-Calib: Automatic LiDAR-Frame/Event Camera Extrinsic Calibration With A Globally Optimal Solution

LiDAR-360 RGB Camera-360 Thermal Camera Targetless Calibration for Dynamic Situations

Extrinsic Calibration between a Camera and a 2D Laser Rangefinder using a Photogrammetric Control Field

Microsoft Azure Kinect Calibration for Three-Dimensional Dense Point Clouds and Reliable Skeletons

A High Precision Approach to Calibrate a Structured Light Vision Sensor in a Robot-Based Three-Dimensional Measurement System

Self-Supervised Camera Self-Calibration from Video

Extrinsic Calibration of Camera and LiDAR Systems With Three‐Dimensional Towered Checkerboards