A comprehensive overview of deep learning techniques for 3D point cloud classification and semantic segmentation

Sushmita Sarker,Prithul Sarker,Gunner Stone,Ryan Gorman,Alireza Tavakkoli,George Bebis,Javad Sattarvand

DOI: https://doi.org/10.1007/s00138-024-01543-1

2024-05-20

Abstract:Point cloud analysis has a wide range of applications in many areas such as computer vision, robotic manipulation, and autonomous driving. While deep learning has achieved remarkable success on image-based tasks, there are many unique challenges faced by deep neural networks in processing massive, unordered, irregular and noisy 3D points. To stimulate future research, this paper analyzes recent progress in deep learning methods employed for point cloud processing and presents challenges and potential directions to advance this field. It serves as a comprehensive review on two major tasks in 3D point cloud processing-- namely, 3D shape classification and semantic segmentation.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper primarily addresses two key issues in 3D point cloud processing: 3D shape classification and semantic segmentation. Specifically, the goals of the paper include:

1. **Comprehensive Review**: Analyze the deep learning methods used in 3D point cloud processing in recent years, discuss the challenges these methods face, and explore future development directions.

2. **Technical Coverage**: Cover the two main tasks of 3D shape classification and semantic segmentation, and go beyond existing reviews by including methods and techniques that were previously under-discussed.

3. **Dataset Overview**: Provide an introduction to several commonly used large datasets for different types of point cloud applications, including datasets for indoor and outdoor scenes.

4. **Evaluation Metrics**: Define the standards for evaluating the performance of algorithms in 3D point cloud applications, such as Overall Accuracy (OA) and mean Class Accuracy (mAcc).

5. **Method Classification**: Classify 3D point cloud classification methods into five main categories: grid-based methods, projection-based methods, voxel-based methods, hybrid methods, and point-based methods, and discuss each method in detail.

6. **Performance Comparison**: Conduct a comprehensive comparison of existing methods using multiple public datasets and discuss the advantages and limitations of various methods in detail.

7. **Future Directions**: Explore the challenges in the current field and provide guidance for future research.

The main contributions of the paper are:

- Summarize the latest deep learning models and techniques for 3D point cloud shape classification and semantic segmentation from 2015 to 2023.
- Go beyond existing literature by covering all existing point cloud classification and segmentation methods, especially those that were previously under-discussed.
- Provide a comprehensive classification system, including supervised and unsupervised methods, as well as previously overlooked grid-based methods.
- Classify and briefly discuss various existing models, enabling readers to understand the specific advantages and application scenarios of these methods.
- Conduct a comprehensive comparison of existing methods using multiple public datasets and discuss the inherent advantages and limitations of these methods in depth.
- Discuss the main challenges faced by the current field and provide insights for future research.

The uniqueness of this review paper lies not only in its coverage of the latest advancements but also in its detailed attention to previously overlooked areas in the literature. Additionally, the structure of the paper makes it an important resource for readers from various backgrounds, benefiting both novices and experts.