Abstract:Convolutional Neural Network (CNN)-based machine learning systems have made breakthroughs in feature extraction and image recognition tasks in two dimensions (2D). Although there is significant ongoing work to apply CNN technology to domains involving complex 3D data, the success of such efforts has been constrained, in part, by limitations in data representation techniques. Most current approaches rely upon low-resolution 3D models, strategic limitation of scope in the 3D space, or the application of lossy projection techniques to allow for the use of 2D CNNs. To address this issue, we present a mapping algorithm that converts 3D structures to 2D and 1D data grids by mapping a traversal of a 3D space-filling curve to the traversal of corresponding 2D and 1D curves. We explore the performance of 2D and 1D CNNs trained on data encoded with our method versus comparable volumetric CNNs operating upon raw 3D data from a popular benchmarking dataset. Our experiments demonstrate that both 2D and 1D representations of 3D data generated via our method preserve a significant proportion of the 3D data's features in forms learnable by CNNs. Furthermore, we demonstrate that our method of encoding 3D data into lower-dimensional representations allows for decreased CNN training time cost, increased original 3D model rendering resolutions, and supports increased numbers of data channels when compared to purely volumetric approaches. This demonstration is accomplished in the context of a structural biology classification task wherein we train 3D, 2D, and 1D CNNs on examples of two homologous branches within the Ras protein family. The essential contribution of this paper is the introduction of a dimensionality-reduction method that may ease the application of powerful deep learning tools to domains characterized by complex structural data.

D eep l earning 3 d s hapes u sing a lt-az a nisotropic 2s phere c onvolution

Deep Learning 3D Shapes Using Alt-az Anisotropic 2-Sphere Convolution

SurfNet: Generating 3D shape surfaces using deep residual networks

Multi-view Convolutional Neural Networks for 3D Shape Recognition

Deep Spherical Panoramic Representation for 3D Shape Recognition

Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction

Local Deep Feature Learning Framework for 3D Shape.

3D-A-Nets: 3D Deep Dense Descriptor for Volumetric Shapes with Adversarial Networks

CNNs on surfaces using rotation-equivariant features

A Spatial Mapping Algorithm with Applications in Deep Learning-Based Structure Classification

Scale-Equivariant Deep Learning for 3D Data

Mesh Convolution with Continuous Filters for 3D Surface Parsing

Anisotropic Multi-Scale Graph Convolutional Network for Dense Shape Correspondence

DeepShape: Deep-Learned Shape Descriptor for 3D Shape Retrieval

DTV-CNN: Neural network based on depth and thickness views for efficient 3D shape classification

A survey of deep learning-based 3D shape generation

An Improved 3D Shape Recognition Method Based on Panoramic View

Progressive Shape-Distribution-Encoder for Learning 3D Shape Representation

Spherical Transformer: Adapting Spherical Signal to CNNs

Deep Marching Cubes: Learning Explicit Surface Representations.