Robust Online Calibration of LiDAR and Camera Based on Cross-Modal Graph Neural Network

Jianxiao Zhu,Xu Li,Qimin Xu,Zhengliang Sun

DOI: https://doi.org/10.1109/tim.2023.3302384

IF: 5.6

2023-01-01

IEEE Transactions on Instrumentation and Measurement

Abstract:Accurate spatial parameters of LiDAR and camera is a prerequisite for information consistency and robust online calibration is the foundation of long-term effective fusion in intelligent perception system. However, dynamic scene conditions and various hardware prior parameters pose great challenges to the robustness and generalization of existing models. To solve these problems, we propose an online calibration method based on a cross-modal graph neural network (GNN). In the data preprocessing stage, the influence of prior parameters on the inductive bias is reduced by a unified spherical space process strategy of 3-D points and 2-D pixels, which strengthens the generalization. In the graph network, the correlation of multiple windows inside the modal and the explicit correlation matrix across the modal are solved by modeling the robust matching process of human visual positioning. In multilevel graphic constraints, the precise relative position and orientation information is obtained by imposing nodes, edges, and embedding constraints on the graph structure. Extensive evaluations on KITTI and PandaSet suggest that the proposed method not only effectively improves the robustness in various scenes, but also enhances the generalization of the online calibration algorithm.