Abstract:Learning discriminative shape representation directly on point clouds is still challenging in 3D shape analysis and understanding. Recent studies usually involve three steps: first splitting a point cloud into some local regions, then extracting the corresponding feature of each local region, and finally aggregating all individual local region features into a global feature as shape representation using simple max-pooling. However, such pooling-based feature aggregation methods do not adequately take the spatial relationships (e.g. the relative locations to other regions) between local regions into account, which greatly limits the ability to learn discriminative shape representation. To address this issue, we propose a novel deep learning network, named Point2SpatialCapsule, for aggregating features and spatial relationships of local regions on point clouds, which aims to learn more discriminative shape representation. Compared with the traditional max-pooling based feature aggregation networks, Point2SpatialCapsule can explicitly learn not only geometric features of local regions but also the spatial relationships among them. Point2SpatialCapsule consists of two main modules. To resolve the disorder problem of local regions, the first module, named geometric feature aggregation, is designed to aggregate the local region features into the learnable cluster centers, which explicitly encodes the spatial locations from the original 3D space. The second module, named spatial relationship aggregation, is proposed for further aggregating the clustered features and the spatial relationships among them in the feature space using the spatial-aware capsules developed in this paper. Compared to the previous capsule network based methods, the feature routing on the spatial-aware capsules can learn more discriminative spatial relationships among local regions for point clouds, which establishes a direct mapping between log priors and the spatial locations through feature clusters. Experimental results demonstrate that Point2SpatialCapsule outperforms the state-of-the-art methods in the 3D shape classification, retrieval and segmentation tasks under the well-known ModelNet and ShapeNet datasets.

Point2SpatialCapsule: Aggregating Features and Spatial Relationships of Local Regions on Point Clouds using Spatial-aware Capsules

3DPointCaps++: Learning 3D Representations with Capsule Networks

Point Cloud Domain Adaptation Via Masked Local 3D Structure Prediction

CapsLoc3D: Point Cloud Retrieval for Large-Scale Place Recognition Based on 3D Capsule Networks

3D Point Capsule Networks

Associate Semantic-Instance Segmentation of 3D Point Clouds Based on Local Feature Extraction

3DCapsule: Extending the Capsule Architecture to Classify 3D Point Clouds

A Transformer-Based Capsule Network for 3D Part–Whole Relationship Learning

Geometric Capsule Autoencoders for 3D Point Clouds

Learning Point Cloud Shapes with Geometric and Topological Structures.

PointSCNet: Point Cloud Structure and Correlation Learning Based on Space Filling Curve-Guided Sampling

LRC-Net: Learning Discriminative Features on Point Clouds by Encoding Local Region Contexts

Background-Aware 3D Point Cloud Segmentationwith Dynamic Point Feature Aggregation

Hybrid Gromov-Wasserstein Embedding for Capsule Learning

Point Cloud Deep Learning Network Based on Local Domain Multi-Level Feature

Point2Sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-Based Sequence to Sequence Network.

DensePoint: Learning Densely Contextual Representation for Efficient Point Cloud Processing

Detail Preserved Point Cloud Completion via Separated Feature Aggregation

PointNAC: Copula-Based Point Cloud Semantic Segmentation Network

Subspace Capsule Network

DE-CapsNet: A Diverse Enhanced Capsule Network with Disperse Dynamic Routing