Abstract:Convolutional Neural Network (CNN)-based machine learning systems have made breakthroughs in feature extraction and image recognition tasks in two dimensions (2D). Although there is significant ongoing work to apply CNN technology to domains involving complex 3D data, the success of such efforts has been constrained, in part, by limitations in data representation techniques. Most current approaches rely upon low-resolution 3D models, strategic limitation of scope in the 3D space, or the application of lossy projection techniques to allow for the use of 2D CNNs. To address this issue, we present a mapping algorithm that converts 3D structures to 2D and 1D data grids by mapping a traversal of a 3D space-filling curve to the traversal of corresponding 2D and 1D curves. We explore the performance of 2D and 1D CNNs trained on data encoded with our method versus comparable volumetric CNNs operating upon raw 3D data from a popular benchmarking dataset. Our experiments demonstrate that both 2D and 1D representations of 3D data generated via our method preserve a significant proportion of the 3D data's features in forms learnable by CNNs. Furthermore, we demonstrate that our method of encoding 3D data into lower-dimensional representations allows for decreased CNN training time cost, increased original 3D model rendering resolutions, and supports increased numbers of data channels when compared to purely volumetric approaches. This demonstration is accomplished in the context of a structural biology classification task wherein we train 3D, 2D, and 1D CNNs on examples of two homologous branches within the Ras protein family. The essential contribution of this paper is the introduction of a dimensionality-reduction method that may ease the application of powerful deep learning tools to domains characterized by complex structural data.

A Convolutional Architecture for 3D Model Embedding

Encoder-Decoder Architecture for 3D Seismic Inversion

Scale-Equivariant Deep Learning for 3D Data

Multi-View 3d Object Retrieval with Deep Embedding Network

Learning Discriminative and Generative Shape Embeddings for Three-Dimensional Shape Retrieval

Multi-view 3D Models from Single Images with a Convolutional Network

DeepVoxels: Learning Persistent 3D Feature Embeddings

A survey on Deep Learning Advances on Different 3D Data Representations

A Spatial Mapping Algorithm with Applications in Deep Learning-Based Structure Classification

Deep Learning Representation using Autoencoder for 3D Shape Retrieval

Maximizing Spatio-Temporal Entropy of Deep 3D CNNs for Efficient Video Recognition

Multiview Compressive Coding for 3D Reconstruction

2L3: Lifting Imperfect Generated 2D Images into Accurate 3D

Three-dimensional reconstruction of an object from a single image using deep convolutional neural networks

Super Images -- A New 2D Perspective on 3D Medical Imaging Analysis

VolumeFusion: Deep Depth Fusion for 3D Scene Reconstruction

Deep Projective 3D Semantic Segmentation

MV-C3D: A Spatial Correlated Multi-View 3D Convolutional Neural Networks

On the Compactness, Efficiency, and Representation of 3D Convolutional Networks: Brain Parcellation As a Pretext Task

Leveraging 2D Deep Learning ImageNet-trained models for Native 3D Medical Image Analysis