Abstract:Current point cloud registration methods predominantly focus on extracting geometric information from point clouds. In certain scenarios, i.e., when the target objects to be registered contain a large number of repetitive planar structures, the point-only based methods struggle to extract distinctive features from the similar structures, which greatly limits the accuracy of registration. Moreover, the deep learning-based approaches achieve commendable results on public datasets, but they face challenges in generalizing to unseen few-shot datasets with significant domain differences from the training data, and that is especially common in industrial applications where samples are generally scarce. Moreover, existing registration methods can achieve high accuracy on complete point clouds. However, for partial point cloud registration, many methods are incapable of accurately identifying correspondences, making it challenging to estimate precise rigid transformations. This paper introduces a domain-adaptive multimodal feature fusion method for partial point cloud registration in an unsupervised manner, named DAMF-Net, that significantly addresses registration challenges in scenes dominated by repetitive planar structures, and it can generalize well-trained networks on public datasets to unseen few-shot datasets. Specifically, we first introduce a point-guided two-stage multimodal feature fusion module that utilizes the geometric information contained in point clouds to guide the texture information in images for preliminary and supplementary feature fusion. Secondly, we incorporate a gradient-inverse domain-aware module to construct a domain classifier in a generative adversarial manner, weakening the feature extractor's ability to distinguish between source and target domain samples, thereby achieving generalization across different domains. Experiments on a public dataset and our industrial components dataset demonstrate that our method improves the registration accuracy in specific scenarios with numerous repetitive planar structures and achieves high accuracy on unseen few-shot datasets, compared with the results of state-of-the-art traditional and deep learning-based point cloud registration methods.

IMFNet: Interpretable Multimodal Fusion for Point Cloud Registration

Boosting 3D Point Cloud Registration by Transferring Multi-modality Knowledge

DAMF-Net: Unsupervised Domain-Adaptive Multimodal Feature Fusion Method for Partial Point Cloud Registration

MAFNet: A two-stage multiple attention fusion network for partial-to-partial point cloud registration

DeepICP: An End-to-End Deep Neural Network for 3D Point Cloud Registration

PointMBF: A Multi-scale Bidirectional Fusion Network for Unsupervised RGB-D Point Cloud Registration

3D Focusing-and-Matching Network for Multi-Instance Point Cloud Registration

HybridFusion: LiDAR and Vision Cross-Source Point Cloud Fusion

Cross-Modal Information-Guided Network using Contrastive Learning for Point Cloud Registration

Self-Supervised Keypoint Detection and Cross-Fusion Matching Networks for Multimodal Remote Sensing Image Registration

MFINet: a multi-scale feature interaction network for point cloud registration

GMF: General Multimodal Fusion Framework for Correspondence Outlier Rejection

DMFF: dual-way multimodal feature fusion for 3D object detection

MFFNet: Multimodal Feature Fusion Network for Point Cloud Semantic Segmentation

Hybrid3D: learning 3D hybrid features with point clouds and multi-view images for point cloud registration

Improving Deep Learning Point Cloud Registration with Semantic Labels

2D3D-MATR: 2D-3D Matching Transformer for Detection-free Registration between Images and Point Clouds

A Principle Design of Registration-Fusion Consistency: Toward Interpretable Deep Unregistered Hyperspectral Image Fusion

Feature Interactive Representation for Point Cloud Registration

PANet: A Point-Attention Based Multi-Scale Feature Fusion Network for Point Cloud Registration