Abstract:Abstract Traditional convolutional neural network (CNN) methods rely on dense tensors, which makes them suboptimal for spatially sparse data. In this paper, we propose a CNN model based on sparse tensors for efficient processing of high-resolution shapes represented as binary voxel occupancy grids. In contrast to a dense CNN that takes the entire voxel grid as input, a sparse CNN processes only on the non-empty voxels, thus reducing the memory and computation overhead caused by the sparse input data. We evaluate our method on two clinically relevant skull reconstruction tasks: (1) given a defective skull, reconstruct the complete skull (i.e., skull shape completion), and (2) given a coarse skull, reconstruct a high-resolution skull with fine geometric details (shape super-resolution). Our method outperforms its dense CNN-based counterparts in the skull reconstruction task quantitatively and qualitatively, while requiring substantially less memory for training and inference. We observed that, on the 3D skull data, the overall memory consumption of the sparse CNN grows approximately linearly during inference with respect to the image resolutions. During training, the memory usage remains clearly below increases in image resolution—an $$\times 8$$ × 8 increase in voxel number leads to less than $$\times 4$$ × 4 increase in memory requirements. Our study demonstrates the effectiveness of using a sparse CNN for skull reconstruction tasks, and our findings can be applied to other spatially sparse problems. We prove this by additional experimental results on other sparse medical datasets, like the aorta and the heart. Project page at https://github.com/Jianningli/SparseCNN .

Fast CNN Inference by Adaptive Sparse Matrix Decomposition.

Deep Neural Network Acceleration with Sparse Prediction Layers

SUBP: Soft Uniform Block Pruning for 1 X N Sparse CNNs Multithreading Acceleration

Adaptive Pixel-wise Structured Sparse Network for Efficient CNNs

Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition

FSCNN: A Fast Sparse Convolution Neural Network Inference System

Accelerating Deep Convnets Via Sparse Subspace Clustering

Sparse Kronecker Canonical Polyadic Decomposition for Convolutional Neural Networks Compression

Adaptive Structured Sparse Network for Efficient CNNs with Feature Regularization.

A GPU-based high-performance optimization method of sparse convolutional neural networks

SparseTrain: Exploiting Dataflow Sparsity for Efficient Convolutional Neural Networks Training

DeltaCNN: End-to-End CNN Inference of Sparse Frame Differences in Videos

Adaptive Structured Sparse Network for Efficient CNNs with Feature Regularization

VSCNN: Convolution Neural Network Accelerator With Vector Sparsity

Sparse convolutional neural network for high-resolution skull shape completion and shape super-resolution

A Unified Approximation Framework for Compressing and Accelerating Deep Neural Networks

Fast Cnn Pruning Via Redundancy-Aware Training

Sparse Convolutional Neural Networks for Medical Image Analysis

An adaptive threshold mechanism for accurate and efficient deep spiking convolutional neural networks

Prune the Convolutional Neural Networks with Sparse Shrink

Exploiting Sparsity to Accelerate Fully Connected Layers of CNN-Based Applications on Mobile SoCs