Abstract:Three-dimensional (3D) point cloud registration is a fundamental step for many 3D modeling and mapping applications. Existing approaches are highly disparate in the data source, scene complexity, and application, therefore the current practices in various point cloud registration tasks are still ad-hoc processes. Recent advances in computer vision and deep learning have shown promising performance in estimating rigid/similarity transformation between unregistered point clouds of complex objects and scenes. However, their performances are mostly evaluated using a limited number of datasets from a single sensor (e.g. Kinect or RealSense cameras), lacking a comprehensive overview of their applicability in photogrammetric 3D mapping scenarios. In this work, we provide a comprehensive review of the state-of-the-art (SOTA) point cloud registration methods, where we analyze and evaluate these methods using a diverse set of point cloud data from indoor to satellite sources. The quantitative analysis allows for exploring the strengths, applicability, challenges, and future trends of these methods. In contrast to existing analysis works that introduce point cloud registration as a holistic process, our experimental analysis is based on its inherent two-step process to better comprehend these approaches including feature/keypoint-based initial coarse registration and dense fine registration through cloud-to-cloud (C2C) optimization. More than ten methods, including classic hand-crafted, deep-learning-based feature correspondence, and robust C2C methods were tested. We observed that the success rate of most of the algorithms are fewer than 40% over the datasets we tested and there are still are large margin of improvement upon existing algorithms concerning 3D sparse corresopondence search, and the ability to register point clouds with complex geometry and occlusions. With the evaluated statistics on three datasets, we conclude the best-performing methods for each step and provide our recommendations, and outlook future efforts.

Image-to-Point Registration Via Cross-Modality Correspondence Retrieval

CFI2P: Coarse-to-Fine Cross-Modal Correspondence Learning for Image-to-Point Cloud Registration

I2P Registration by Learning the Underlying Alignment Feature Space from Pixel-to-Point Similarities

HybridFusion: LiDAR and Vision Cross-Source Point Cloud Fusion

Differentiable Registration of Images and LiDAR Point Clouds with VoxelPoint-to-Pixel Matching

End-to-End 3D Point Cloud Learning for Registration Task Using Virtual Correspondences.

OL-Reg: Registration of Image and Sparse LiDAR Point Cloud with Object-Level Dense Correspondences

Point cloud registration for LiDAR and photogrammetric data: A critical synthesis and performance analysis on classic and deep learning algorithms

Boosting 3D Point Cloud Registration by Transferring Multi-modality Knowledge

FreeReg: Image-to-Point Cloud Registration Leveraging Pretrained Diffusion Models and Monocular Depth Estimators

Efficient and Deterministic Search Strategy Based on Residual Projections for Point Cloud Registration with Correspondences

Street view cross-sourced point cloud matching and registration

Quantity-Aware Coarse-to-Fine Correspondence for Image-to-Point Cloud Registration

Fast and Robust Registration of Aerial Images and LiDAR data Based on Structrual Features and 3D Phase Correlation

Fast Registration of Terrestrial LiDAR Point Clouds Based on Gaussian-Weighting Projected Image Matching

2D3D-MATR: 2D-3D Matching Transformer for Detection-free Registration between Images and Point Clouds

A Novel Method for Registration of MLS and Stereo Reconstructed Point Clouds

DeepI2P: Image-to-Point Cloud Registration via Deep Classification

Automatic Multi-View Registration of Point Clouds Via a High-Quality Descriptor and a Novel 3D Transformation Estimation Technique

PHOTOGRAMMETRY AND COMPUTED TOMOGRAPHY POINT CLOUD REGISTRATION USING VIRTUAL CONTROL POINTS

A Registration Method for Large-Angle PointClouds