Semantic segmentation of 3D indoor LiDAR point clouds through feature pyramid architecture search

Haojia Lin,Shangbin Wu,Yiping Chen,Wen Li,Zhipeng Luo,Yulan Guo,Cheng Wang,Jonathan Li

DOI: https://doi.org/10.1016/j.isprsjprs.2021.05.009

IF: 12.7

2021-07-01

ISPRS Journal of Photogrammetry and Remote Sensing

Abstract:<p>Semantic segmentation of 3D Light Detection and Ranging (LiDAR) indoor point clouds using deep learning has been an active topic in recent years. However, most deep neural networks on point clouds conduct multi-level feature fusion via a simple U-shape architecture, which lacks enough capacity on both classification and localization in the segmentation task. In this paper, we propose a Neural Architecture Search (NAS) method to search a Feature Pyramid Network (FPN) module for 3D indoor point cloud semantic segmentation. Specifically, we aim to automatically find an effective feature pyramid architecture as a feature fusion neck in a designed novel pyramidal search space covering all information communication paths for multi-level features. The searched FPN module, named SFPN, contains the most important connections among all the potential paths to fuse representations at different levels. Our proposed SFPN is generic and effective as well as capable to be added to existing segmentation networks to augment the segmentation performance. Extensive experiments on ScanNet and S3DIS show that consistent and remarkable gains of segmentation performance can be achieved by different classical networks combined with SFPN. Specially, PointNet++-SFPN achieves mIoU gains of 7.8% on ScanNet v2 and 4.7% on S3DIS, and PointConv-SFPN achieves 4.5% and 3.7% improvement respectively on the above datasets.</p>

imaging science & photographic technology,remote sensing,geography, physical,geosciences, multidisciplinary

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that in the semantic segmentation task of 3D indoor LiDAR point clouds, when existing deep neural networks perform multi - level feature fusion through a simple U - shaped architecture, they have insufficient classification and localization capabilities. To improve these capabilities, the author proposes a method based on Neural Architecture Search (NAS) to find an effective Feature Pyramid Network (FPN) module for the semantic segmentation of 3D indoor point clouds. Specifically, the goal of the paper is to automatically find an effective feature pyramid architecture as the "neck" part of feature fusion, which can cover all information communication paths of multi - level features. The proposed SFPN (Searched Feature Pyramid Network) module is not only universal and effective, but can also be added to existing segmentation networks to enhance segmentation performance. Through extensive experiments on the ScanNet and S3DIS datasets, it has been proven that after different classical networks are combined with SFPN, the segmentation performance has been significantly improved.