Abstract:Self-supervised learning has not been fully explored for point cloud analysis. Current frameworks are mainly based on point cloud reconstruction. Given only 3D coordinates, such approaches tend to learn local geometric structures and contours, while failing in understanding high level semantic content. Consequently, they achieve unsatisfactory performance in downstream tasks such as classification, segmentation, etc. To fill this gap, we propose a generic Contour-Perturbed Reconstruction Network (CP-Net), which can effectively guide self-supervised reconstruction to learn semantic content in the point cloud, and thus promote discriminative power of point cloud representation. First, we introduce a concise contour-perturbed augmentation module for point cloud reconstruction. With guidance of geometry disentangling, we divide point cloud into contour and content components. Subsequently, we perturb the contour components and preserve the content components on the point cloud. As a result, self supervisor can effectively focus on semantic content, by reconstructing the original point cloud from such perturbed one. Second, we use this perturbed reconstruction as an assistant branch, to guide the learning of basic reconstruction branch via a distinct dual-branch consistency loss. In this case, our CP-Net not only captures structural contour but also learn semantic content for discriminative downstream tasks. Finally, we perform extensive experiments on a number of point cloud benchmarks. Part segmentation results demonstrate that our CP-Net (81.5% of mIoU) outperforms the previous self-supervised models, and narrows the gap with the fully-supervised methods. For classification, we get a competitive result with the fully-supervised methods on ModelNet40 (92.5% accuracy) and ScanObjectNN (87.9% accuracy). The codes and models will be released afterwards.

3D Reconstruction and Semantic Segmentation Method Combining PointNet and 3D-Lmnet from Single Image

3D Point Cloud Generation Reconstruction from Single Image Based on Image Retrieval

Associate Semantic-Instance Segmentation of 3D Point Clouds Based on Local Feature Extraction

Semantic Reconstruction based on RGB Image and Sparse Depth

3D-Isrnet

Point Scene Understanding Via Points-to-mesh Reconstruction and Multi-Level Utilization of Proposals

A New Semantic Segmentation Method of Point Cloud Based on PointNet and VoxelNet

A Single Stage and Single View 3D Point Cloud Reconstruction Network Based on DetNet

Semantic-assisted Unified Network for Feature Point Extraction and Matching

3D-PSRNet: Part Segmented 3D Point Cloud Reconstruction From a Single Image

3D Point Cloud of Single Tree Branches and Leaves Semantic Segmentation Based on Modified PointNet Network

Three-Dimensional Point Cloud Semantic Segmentation Network Based on Spatial Graph Convolution Network

3D-LMNet: Latent Embedding Matching for Accurate and Diverse 3D Point Cloud Reconstruction from a Single Image

MInet: A Novel Network Model for Point Cloud Processing by Integrating Multi-Modal Information

PointNAC: Copula-Based Point Cloud Semantic Segmentation Network

3D Scene Reconstruction with Sparse LiDAR Data and Monocular Image in Single Frame

LAM3D: Large Image-Point-Cloud Alignment Model for 3D Reconstruction from Single Image

Collaborative Network for Super-Resolution and Semantic Segmentation of Remote Sensing Images

CP-Net: Contour-Perturbed Reconstruction Network for Self-Supervised Point Cloud Learning

PointNest: Learning Deep Multiscale Nested Feature Propagation for Semantic Segmentation of 3-D Point Clouds

JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D Point Clouds