Abstract:Recent advances in point cloud registration mostly leverage geometric information. Although these methods have yielded promising results, they still struggle with problems of low overlap, thus limiting their practical usage. In this paper, we propose ML-SemReg, a plug-and-play point cloud registration framework that fully exploits semantic information. Our key insight is that mismatches can be categorized into two types, i.e., inter- and intra-class, after rendering semantic clues, and can be well addressed by utilizing multi-level semantic consistency. We first propose a Group Matching module to address inter-class mismatching, outputting multiple matching groups that inherently satisfy Local Semantic Consistency. For each group, a Mask Matching module based on Scene Semantic Consistency is then introduced to suppress intra-class mismatching. Benefit from those two modules, ML-SemReg generates correspondences with a high inlier ratio. Extensive experiments demonstrate excellent performance and robustness of ML-SemReg, e.g., in hard-cases of the KITTI dataset, the Registration Recall of MAC increases by almost 34 percentage points when our ML-SemReg is equipped. Code is available at \url{<a class="link-external link-https" href="https://github.com/Laka-3DV/ML-SemReg" rel="external noopener nofollow">this https URL</a>}

What problem does this paper attempt to address?

This paper attempts to address the issue of low overlap rate in Point Cloud Registration (PCR). Although existing geometry-based methods have made significant progress in point cloud registration, they still face challenges when dealing with low overlap rates. This paper proposes a framework called ML-SemReg, which improves registration performance by fully utilizing multi-level semantic consistency. ### Main Issues 1. **Low Overlap Rate Issue**: Existing methods perform poorly when dealing with point clouds with low overlap rates, limiting their practical applications. 2. **Mismatching Issue**: Local geometric similarity in point clouds leads to two types of mismatches: inter-class mismatching and intra-class mismatching. ### Solutions 1. **Multi-level Semantic Consistency**: - **Local Semantic Consistency (LS-Consistency)**: Introduces Local Semantic Signatures (Local-SS) to identify local semantic relationships between key points. - **Scene Semantic Consistency (SS-Consistency)**: Constructs Binary Multi-Ring Semantic Signatures (BMR-SS) to capture scene semantic information. 2. **Module Design**: - **Group Matching Module**: Classifies key points into matching groups, ensuring that key points within each group satisfy local semantic consistency, thereby reducing inter-class mismatching. - **Mask Matching Module**: Within each matching group, suppresses intra-class mismatching through scene semantic consistency. ### Experimental Results - **Datasets**: Uses the KITTI dataset for outdoor scenes and the ScanNet dataset for indoor scenes. - **Performance Improvement**: Experimental results show that ML-SemReg significantly improves registration recall (RR) and registration accuracy (RE, TE) across various benchmarks. It performs particularly well in low overlap rate scenarios. ### Contributions 1. **Group Matching Module**: Addresses inter-class mismatching through local semantic consistency. 2. **Mask Matching Module**: Addresses intra-class mismatching through scene semantic consistency. 3. **Plug-in Framework**: ML-SemReg can be integrated as a plug-in into existing point cloud registration methods, ensuring multi-level semantic consistency and generating high-quality correspondences. ### Conclusion The ML-SemReg framework proposed in this paper effectively addresses the issues of low overlap rate and mismatching in point cloud registration by fully utilizing multi-level semantic information, significantly improving registration performance and robustness.

ML-SemReg: Boosting Point Cloud Registration with Multi-level Semantic Consistency

Improving Deep Learning Point Cloud Registration with Semantic Labels

Semantics and Spatial Structures Guided Point Cloud Registration

Semantic enhancement based adaptive geometric encoding network for low overlap point cloud registration

Fast Semantic-Assisted Outlier Removal for Large-scale Point Cloud Registration

PointMS: Semantic Segmentation for Point Cloud Based on Multi-scale Directional Convolution

Boosting 3D Point Cloud Registration by Transferring Multi-modality Knowledge

Robust Point Set Registration Based on Semantic Information

Using Multi-Level Consistency Learning for Partial-to-Partial Point Cloud Registration.

Integrating Deep Semantic Segmentation Into 3-D Point Cloud Registration

SGNet: Salient Geometric Network for Point Cloud Registration

Deep Semantic Graph Matching for Large-scale Outdoor Point Clouds Registration

Deep Semantic Graph Matching for Large-Scale Outdoor Point Cloud Registration

A Novel Method for Registration of MLS and Stereo Reconstructed Point Clouds

Cross-Modal Information-Guided Network using Contrastive Learning for Point Cloud Registration

A Robust Multi-Local to Global with Outlier Filtering for Point Cloud Registration

Geometry-Aware Enhanced Mutual-Supervised Point Elimination with Overlapping Mask Contrastive Learning for Partitial Point Cloud Registration

Multi-Model 3D Registration: Finding Multiple Moving Objects in Cluttered Point Clouds

SGFeat: Salient Geometric Feature for Point Cloud Registration

SGOR: Outlier Removal by Leveraging Semantic and Geometric Information for Robust Point Cloud Registration