Abstract:Learning discriminative shape representation directly on point clouds is still challenging in 3D shape analysis and understanding. Recent studies usually involve three steps: first splitting a point cloud into some local regions, then extracting the corresponding feature of each local region, and finally aggregating all individual local region features into a global feature as shape representation using simple max-pooling. However, such pooling-based feature aggregation methods do not adequately take the spatial relationships (e.g. the relative locations to other regions) between local regions into account, which greatly limits the ability to learn discriminative shape representation. To address this issue, we propose a novel deep learning network, named Point2SpatialCapsule, for aggregating features and spatial relationships of local regions on point clouds, which aims to learn more discriminative shape representation. Compared with the traditional max-pooling based feature aggregation networks, Point2SpatialCapsule can explicitly learn not only geometric features of local regions but also the spatial relationships among them. Point2SpatialCapsule consists of two main modules. To resolve the disorder problem of local regions, the first module, named geometric feature aggregation, is designed to aggregate the local region features into the learnable cluster centers, which explicitly encodes the spatial locations from the original 3D space. The second module, named spatial relationship aggregation, is proposed for further aggregating the clustered features and the spatial relationships among them in the feature space using the spatial-aware capsules developed in this paper. Compared to the previous capsule network based methods, the feature routing on the spatial-aware capsules can learn more discriminative spatial relationships among local regions for point clouds, which establishes a direct mapping between log priors and the spatial locations through feature clusters. Experimental results demonstrate that Point2SpatialCapsule outperforms the state-of-the-art methods in the 3D shape classification, retrieval and segmentation tasks under the well-known ModelNet and ShapeNet datasets.

Deep Spatiality: Unsupervised Learning of Spatially-Enhanced Global and Local 3D Features by Deep Neural Network With Coupled Softmax.

Hamming Embedding Sensitivity Guided Fusion Network for 3D Shape Representation.

Local Deep Feature Learning Framework for 3D Shape.

A Unified Feature Representation and Learning Framework for 3D Shape

Mesh Convolutional Restricted Boltzmann Machines for Unsupervised Learning of Features With Structure Preservation on 3-D Meshes.

Unsupervised Learning of 3-D Local Features from Raw Voxels Based on a Novel Permutation Voxelization Strategy

Multi-Angle Point Cloud-VAE: Unsupervised Feature Learning for 3D Point Clouds from Multiple Angles by Joint Self-Reconstruction and Half-to-Half Prediction

Exploring Deep 3D Spatial Encodings for Large-Scale 3D Scene Understanding

Spatially-Aware Context Neural Networks.

A Dimensional Reduction Guiding Deep Learning Architecture for 3D Shape Retrieval

Learning Point Cloud Shapes with Geometric and Topological Structures.

Learning High-Level Feature by Deep Belief Networks for 3-D Model Retrieval and Recognition.

A Spatial Mapping Algorithm with Applications in Deep Learning-Based Structure Classification

Deep Learning Representation using Autoencoder for 3D Shape Retrieval

BoSCC: Bag of Spatial Context Correlations for Spatially Enhanced 3D Shape Representation

Deep Learning 3D Shapes Using Alt-az Anisotropic 2-Sphere Convolution

DeepSSN: A deep convolutional neural network to assess spatial scene similarity

Deep Volumetric Descriptor Learning for Dense Correspondence of Cone-Beam Computed Tomography Via Spectral Maps

Parts4Feature: Learning 3D Global Features from Generally Semantic Parts in Multiple Views

Point2SpatialCapsule: Aggregating Features and Spatial Relationships of Local Regions on Point Clouds using Spatial-aware Capsules

Challenge of Spatial Cognition for Deep Learning