Abstract:Convolutional Neural Network (CNN)-based machine learning systems have made breakthroughs in feature extraction and image recognition tasks in two dimensions (2D). Although there is significant ongoing work to apply CNN technology to domains involving complex 3D data, the success of such efforts has been constrained, in part, by limitations in data representation techniques. Most current approaches rely upon low-resolution 3D models, strategic limitation of scope in the 3D space, or the application of lossy projection techniques to allow for the use of 2D CNNs. To address this issue, we present a mapping algorithm that converts 3D structures to 2D and 1D data grids by mapping a traversal of a 3D space-filling curve to the traversal of corresponding 2D and 1D curves. We explore the performance of 2D and 1D CNNs trained on data encoded with our method versus comparable volumetric CNNs operating upon raw 3D data from a popular benchmarking dataset. Our experiments demonstrate that both 2D and 1D representations of 3D data generated via our method preserve a significant proportion of the 3D data's features in forms learnable by CNNs. Furthermore, we demonstrate that our method of encoding 3D data into lower-dimensional representations allows for decreased CNN training time cost, increased original 3D model rendering resolutions, and supports increased numbers of data channels when compared to purely volumetric approaches. This demonstration is accomplished in the context of a structural biology classification task wherein we train 3D, 2D, and 1D CNNs on examples of two homologous branches within the Ras protein family. The essential contribution of this paper is the introduction of a dimensionality-reduction method that may ease the application of powerful deep learning tools to domains characterized by complex structural data.

Deep Learning 3D Shapes Using Alt-az Anisotropic 2-Sphere Convolution

D eep l earning 3 d s hapes u sing a lt-az a nisotropic 2s phere c onvolution

Local Deep Feature Learning Framework for 3D Shape.

Deep Spherical Panoramic Representation for 3D Shape Recognition

3D-A-Nets: 3D Deep Dense Descriptor for Volumetric Shapes with Adversarial Networks

A Dimensional Reduction Guiding Deep Learning Architecture for 3D Shape Retrieval

Multi-view Convolutional Neural Networks for 3D Shape Recognition

Deep Learning Representation using Autoencoder for 3D Shape Retrieval

SurfNet: Generating 3D shape surfaces using deep residual networks

Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction

A survey of deep learning-based 3D shape generation

DeepShape: Deep-Learned Shape Descriptor for 3D Shape Retrieval

A new 3D shape descriptor based on rotation

Anisotropic Multi-Scale Graph Convolutional Network for Dense Shape Correspondence

Learning Point Cloud Shapes with Geometric and Topological Structures.

An Improved 3D Shape Recognition Method Based on Panoramic View

DTV-CNN: Neural network based on depth and thickness views for efficient 3D shape classification

Learning High-Level Feature by Deep Belief Networks for 3-D Model Retrieval and Recognition.

A Spatial Mapping Algorithm with Applications in Deep Learning-Based Structure Classification

Exploring Deep 3D Spatial Encodings for Large-Scale 3D Scene Understanding