Abstract:LiDAR and camera are complementary sensors for 3D object detection in autonomous driving. However, it is challenging to explore the unnatural interaction between point clouds and images, and the critical factor is how to conduct feature alignment of these heterogeneous modalities. Currently, many methods achieve feature alignment through projection calibration, without accounting for the impact of sensors misalignment errors, resulting in sub-optimal performance. In this paper, we present GraphAlign++, a more accurate feature alignment framework for 3D object detection by graph matching. Specifically, we construct the nearest neighbor relationship by calculating Euclidean distances of point cloud features within the subspaces. Through the projection calibration between the image and point cloud pairs, we project the nearest neighbors of point cloud features onto the corresponding image. Then by matching the nearest neighbors of a single point-feature of the point cloud with multiple pixel-features of the image, we search for a more appropriate feature alignment. In addition, we provide a self-attention module to enhance the weights of significant relations to fine-tune the feature alignment between these two heterogeneous modalities. Extensive experiments on nuScenes benchmark demonstrate the effectiveness and efficiency of GraphAlign++. Notably, due to the more accurate feature alignment, which contributes to increase mAP by 3.10% on KITTI test hard level, our method is remarkably beneficial for long-range object detection.

A General Framework for 3D Model Co-Alignment.

Direct Alignment with Generalized Correspondences: A Unified Framework for Structure-Based Visual Pose Estimation.

3D Shape Matching Using Collinearity Constraint

Face Alignment Based on 3D Face Shape Model and Markov Random Field.

Source-enhanced Prototypical Alignment for Single Image 3D Model Retrieval

Face alignment based on high order markov random field

A Unified Feature Representation and Learning Framework for 3D Shape

On the Essence and Prospect: An Investigation of Alignment Approaches for Big Models

GraphAlign: Enhancing Accurate Feature Alignment by Graph matching for Multi-Modal 3D Object Detection

Generic Face Alignment Using an Improved Active Shape Model

Automatic Feature-Based Point Cloud Alignment and Inspection

GraphAlign++: An Accurate Feature Alignment by Graph Matching for Multi-Modal 3D Object Detection

Unbiased Feature Position Alignment for Human Pose Estimation

Scalable unsupervised alignment of general metric and non-metric structures

Face Alignment Across Large Poses: A 3D Solution

Manifold Alignment Via Corresponding Projections.

Robust Collaborative 3D Object Detection in Presence of Pose Errors.

3D Face Alignment Through Fusion of Head Pose Information and Features

A survey and classification of face alignment methods based on face models

Modeling Multi-mapping Relations for Precise Cross-lingual Entity Alignment

Image-Based Alignment of 3D Scans