Abstract:Learning discriminative shape representation directly on point clouds is still challenging in 3D shape analysis and understanding. Recent studies usually involve three steps: first splitting a point cloud into some local regions, then extracting the corresponding feature of each local region, and finally aggregating all individual local region features into a global feature as shape representation using simple max-pooling. However, such pooling-based feature aggregation methods do not adequately take the spatial relationships (e.g. the relative locations to other regions) between local regions into account, which greatly limits the ability to learn discriminative shape representation. To address this issue, we propose a novel deep learning network, named Point2SpatialCapsule, for aggregating features and spatial relationships of local regions on point clouds, which aims to learn more discriminative shape representation. Compared with the traditional max-pooling based feature aggregation networks, Point2SpatialCapsule can explicitly learn not only geometric features of local regions but also the spatial relationships among them. Point2SpatialCapsule consists of two main modules. To resolve the disorder problem of local regions, the first module, named geometric feature aggregation, is designed to aggregate the local region features into the learnable cluster centers, which explicitly encodes the spatial locations from the original 3D space. The second module, named spatial relationship aggregation, is proposed for further aggregating the clustered features and the spatial relationships among them in the feature space using the spatial-aware capsules developed in this paper. Compared to the previous capsule network based methods, the feature routing on the spatial-aware capsules can learn more discriminative spatial relationships among local regions for point clouds, which establishes a direct mapping between log priors and the spatial locations through feature clusters. Experimental results demonstrate that Point2SpatialCapsule outperforms the state-of-the-art methods in the 3D shape classification, retrieval and segmentation tasks under the well-known ModelNet and ShapeNet datasets.

3DPointCaps++: Learning 3D Representations with Capsule Networks

3D Point Capsule Networks

Point2SpatialCapsule: Aggregating Features and Spatial Relationships of Local Regions on Point Clouds using Spatial-aware Capsules

Geometric Capsule Autoencoders for 3D Point Clouds

CapsLoc3D: Point Cloud Retrieval for Large-Scale Place Recognition Based on 3D Capsule Networks

3DCapsule: Extending the Capsule Architecture to Classify 3D Point Clouds

DeepCaps: Going Deeper With Capsule Networks

Hierarchical Object-Centric Learning with Capsule Networks

DE-CapsNet: A Diverse Enhanced Capsule Network with Disperse Dynamic Routing

Quaternion Equivariant Capsule Networks for 3D Point Clouds

A Transformer-Based Capsule Network for 3D Part–Whole Relationship Learning

CapProNet: Deep Feature Learning via Orthogonal Projections onto Capsule Subspaces

Adaptive Capsule Network

Capsule Networks With Residual Pose Routing

A lightweight capsule network via channel-space decoupling and self-attention routing

Orthogonal Capsule Networks With Positional Information Preservation and Lightweight Feature Learning

Hybrid Gromov-Wasserstein Embedding for Capsule Learning

SS-3DCapsNet: Self-supervised 3D Capsule Networks for Medical Segmentation on Less Labeled Data

Object-centric Learning with Capsule Networks: A Survey

Graphics Capsule: Learning Hierarchical 3D Face Representations from 2D Images

Spatiality-guided Transformer for 3D Dense Captioning on Point Clouds