Abstract:Convolutional Neural Network (CNN)-based machine learning systems have made breakthroughs in feature extraction and image recognition tasks in two dimensions (2D). Although there is significant ongoing work to apply CNN technology to domains involving complex 3D data, the success of such efforts has been constrained, in part, by limitations in data representation techniques. Most current approaches rely upon low-resolution 3D models, strategic limitation of scope in the 3D space, or the application of lossy projection techniques to allow for the use of 2D CNNs. To address this issue, we present a mapping algorithm that converts 3D structures to 2D and 1D data grids by mapping a traversal of a 3D space-filling curve to the traversal of corresponding 2D and 1D curves. We explore the performance of 2D and 1D CNNs trained on data encoded with our method versus comparable volumetric CNNs operating upon raw 3D data from a popular benchmarking dataset. Our experiments demonstrate that both 2D and 1D representations of 3D data generated via our method preserve a significant proportion of the 3D data's features in forms learnable by CNNs. Furthermore, we demonstrate that our method of encoding 3D data into lower-dimensional representations allows for decreased CNN training time cost, increased original 3D model rendering resolutions, and supports increased numbers of data channels when compared to purely volumetric approaches. This demonstration is accomplished in the context of a structural biology classification task wherein we train 3D, 2D, and 1D CNNs on examples of two homologous branches within the Ras protein family. The essential contribution of this paper is the introduction of a dimensionality-reduction method that may ease the application of powerful deep learning tools to domains characterized by complex structural data.

3D reconstruction of curvilinear structures with stereo matching deep convolutional neural networks

3D Reconstruction of Curvilinear Structures with Stereo Matching DeepConvolutional Neural Networks

Using convolutional neural networks for stereological characterization of 3D hetero-aggregates based on synthetic STEM data

StrainNet-3D: Real-time and Robust 3-Dimensional Speckle Image Correlation Using Deep Learning

Missing Wedge Completion via Unsupervised Learning with Coordinate Networks

Three dimensional classification of dislocations from single projections

Disentangling multiple scattering with deep learning: application to strain mapping from electron diffraction patterns

Adaptive 3D convolutional neural network-based reconstruction method for 3D coherent diffraction imaging

Direct recognition of crystal structures via three-dimensional convolutional neural networks with high accuracy and tolerance to random displacements and missing atoms

Toward 3D Object Reconstruction from Stereo Images

3D Structure from 2D Microscopy images using Deep Learning

Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation

Joint Cryo-ET Alignment and Reconstruction with Neural Deformation Fields

A binocular stereo-view 3D reconstruction algorithm based on contour extraction and depth screening

Semi-Dense 3D Reconstruction with a Stereo Event Camera

A Spatial Mapping Algorithm with Applications in Deep Learning-Based Structure Classification

Deep Learning of Structural Morphology Imaged by Scanning X-ray Diffraction Microscopy

Fast 3D reconstruction via event-based structured light with spatio-temporal coding

Polka Lines: Learning Structured Illumination and Reconstruction for Active Stereo

Deep Stereo Matching with Dense CRF Priors