Abstract:Convolutional Neural Network (CNN)-based machine learning systems have made breakthroughs in feature extraction and image recognition tasks in two dimensions (2D). Although there is significant ongoing work to apply CNN technology to domains involving complex 3D data, the success of such efforts has been constrained, in part, by limitations in data representation techniques. Most current approaches rely upon low-resolution 3D models, strategic limitation of scope in the 3D space, or the application of lossy projection techniques to allow for the use of 2D CNNs. To address this issue, we present a mapping algorithm that converts 3D structures to 2D and 1D data grids by mapping a traversal of a 3D space-filling curve to the traversal of corresponding 2D and 1D curves. We explore the performance of 2D and 1D CNNs trained on data encoded with our method versus comparable volumetric CNNs operating upon raw 3D data from a popular benchmarking dataset. Our experiments demonstrate that both 2D and 1D representations of 3D data generated via our method preserve a significant proportion of the 3D data's features in forms learnable by CNNs. Furthermore, we demonstrate that our method of encoding 3D data into lower-dimensional representations allows for decreased CNN training time cost, increased original 3D model rendering resolutions, and supports increased numbers of data channels when compared to purely volumetric approaches. This demonstration is accomplished in the context of a structural biology classification task wherein we train 3D, 2D, and 1D CNNs on examples of two homologous branches within the Ras protein family. The essential contribution of this paper is the introduction of a dimensionality-reduction method that may ease the application of powerful deep learning tools to domains characterized by complex structural data.

Sparse 3D convolutional neural networks

Lightweight Image Super-Resolution Network Using 3D Convolutional Neural Networks

Convolutional Neural Network Based Computational Imaging Spectroscopy

Sparse Convolutional Neural Networks for Medical Image Analysis

Sparse convolutional neural network for high-resolution skull shape completion and shape super-resolution

Spatial Pruned Sparse Convolution for Efficient 3D Object Detection

SparseVoxNet: 3-D Object Recognition With Sparsely Aggregation of 3-D Dense Blocks

Putting 3D Spatially Sparse Networks on a Diet

Optimizing Sparse Convolution on GPUs with CUDA for 3D Point Cloud Processing in Embedded Systems

Dynamic Spatio-Temporal Feature Learning via Graph Convolution in 3D Convolutional Networks

Sparsity Invariant CNNs

Unsupervised learning with sparse space-and-time autoencoders

Scalable Deep Convolutional Neural Networks for Sparse, Locally Dense Liquid Argon Time Projection Chamber Data

SPLATNet: Sparse Lattice Networks for Point Cloud Processing

A Scalable 3D Array Architecture for Accelerating Convolutional Neural Networks

Efficient 3D Recognition with Event-driven Spike Sparse Convolution

High Performance Implementation of 3D Convolutional Neural Networks on a GPU

SparseTrain: Exploiting Dataflow Sparsity for Efficient Convolutional Neural Networks Training

A Spatial Mapping Algorithm with Applications in Deep Learning-Based Structure Classification

Frequency Domain Compact 3d Convolutional Neural Networks

SRCN3D: Sparse R-CNN 3D for Compact Convolutional Multi-View 3D Object Detection and Tracking