Comparison of Point Cloud Registration Techniques on Scanned Physical Objects

Menthy Denayer,Joris De Winter,Evandro Bernardes,Bram Vanderborght,Tom Verstraten
DOI: https://doi.org/10.3390/s24072142
IF: 3.9
2024-03-28
Sensors
Abstract:This paper presents a comparative analysis of six prominent registration techniques for solving CAD model alignment problems. Unlike the typical approach of assessing registration algorithms with synthetic datasets, our study utilizes point clouds generated from the Cranfield benchmark. Point clouds are sampled from existing CAD models and 3D scans of physical objects, introducing real-world complexities such as noise and outliers. The acquired point cloud scans, including ground-truth transformations, are made publicly available. This dataset includes several cleaned-up scans of nine 3D-printed objects. Our main contribution lies in assessing the performance of three classical (GO-ICP, RANSAC, FGR) and three learning-based (PointNetLK, RPMNet, ROPNet) methods on real-world scans, using a wide range of metrics. These include recall, accuracy and computation time. Our comparison shows a high accuracy for GO-ICP, as well as PointNetLK, RANSAC and RPMNet combined with ICP refinement. However, apart from GO-ICP, all methods show a significant number of failure cases when applied to scans containing more noise or requiring larger transformations. FGR and RANSAC are among the quickest methods, while GO-ICP takes several seconds to solve. Finally, while learning-based methods demonstrate good performance and low computation times, they have difficulties in training and generalizing. Our results can aid novice researchers in the field in selecting a suitable registration method for their application, based on quantitative metrics. Furthermore, our code can be used by others to evaluate novel methods.
engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
What problem does this paper attempt to address?
This paper attempts to solve the problem of performance evaluation of point cloud registration techniques in practical applications. Specifically, the author focuses on how to effectively compare the performance of different point cloud registration methods when processing real - world scan data. Traditional evaluation methods usually use synthetic datasets to test registration algorithms, but these synthetic datasets are often unable to fully simulate the complex situations in the real world, such as noise, partiality, and deformation. Therefore, this paper evaluates six popular point cloud registration techniques, including three classical methods (GO - ICP, RANSAC, FGR) and three learning - based methods (PointNetLK, RPMNet, ROPNet), by using real - point - cloud - scan data generated from the Cranfield benchmark dataset. The performance of these methods when processing real - scan data containing noise and requiring large transformations is evaluated in detail to provide useful guidance for novice researchers and an open - source dataset and code framework for future research. ### Main Problems 1. **Complexity of Real - World Data**: Existing evaluation methods mostly rely on synthetic datasets, and these datasets are often unable to fully reflect complex situations in the real world such as noise, partiality, and deformation. 2. **Performance Comparison of Different Registration Methods**: There is a lack of systematic comparison of multiple registration methods on real - world data, especially the comparison between classical methods and learning - based methods. 3. **Open - Source Datasets and Codes**: There is a lack of publicly accessible datasets and evaluation codes containing real - world point - cloud - scan data in the existing literature, which limits further verification and improvement by researchers. ### Solutions 1. **Using Real - World Data**: Generate real - point - cloud - scan data containing noise and partiality through the Cranfield benchmark dataset for evaluating registration methods. 2. **Multi - Index Evaluation**: Adopt multiple indexes including recall rate, accuracy rate, and calculation time to comprehensively evaluate the performance of different registration methods. 3. **Open - Source Datasets and Codes**: Provide a dataset containing real - point - cloud - scan data and its ground truth, and publish the experimental code for other researchers to verify and expand the research. ### Main Contributions 1. **Performance Evaluation**: A detailed comparison of the performance of six registration methods on real - world scan data is made, providing quantitative evaluation results. 2. **Datasets and Codes**: A dataset containing real - point - cloud - scan data is created, and Python code for evaluation is provided, facilitating other researchers to reproduce and expand the research. 3. **Guiding Significance**: Provide guidance for novice researchers to select registration methods suitable for their application scenarios based on quantitative evaluation indexes. Through these efforts, the paper not only fills the gaps in existing research but also provides an important foundation and tools for future research.