Abstract:The study of sparsity in Convolutional Neural Networks (CNNs) has become widespread to compress and accelerate models in environments with limited resources. By constraining N consecutive weights along the output channel to be group-wise non-zero, the recent network with 1$\times$N sparsity has received tremendous popularity for its three outstanding advantages: 1) A large amount of storage space saving by a \emph{Block Sparse Row} matrix. 2) Excellent performance at a high sparsity. 3) Significant speedups on CPUs with Advanced Vector Extensions. Recent work requires selecting and fine-tuning 1$\times$N sparse weights based on dense pre-trained weights, leading to the problems such as expensive training cost and memory access, sub-optimal model quality, as well as unbalanced workload across threads (different sparsity across output channels). To overcome them, this paper proposes a novel \emph{\textbf{S}oft \textbf{U}niform \textbf{B}lock \textbf{P}runing} (SUBP) approach to train a uniform 1$\times$N sparse structured network from scratch. Specifically, our approach tends to repeatedly allow pruned blocks to regrow to the network based on block angular redundancy and importance sampling in a uniform manner throughout the training process. It not only makes the model less dependent on pre-training, reduces the model redundancy and the risk of pruning the important blocks permanently but also achieves balanced workload. Empirically, on ImageNet, comprehensive experiments across various CNN architectures show that our SUBP consistently outperforms existing 1$\times$N and structured sparsity methods based on pre-trained models or training from scratch. Source codes and models are available at \url{https://github.com/JingyangXiang/SUBP}.

Joint Sparsity with Mixed Granularity for Efficient GPU Implementation

Deep Neural Network Acceleration with Sparse Prediction Layers

Balanced Sparsity for Efficient DNN Inference on GPU

Regularization-Free Structural Pruning for GPU Inference Acceleration

SUBP: Soft Uniform Block Pruning for 1xn Sparse CNNs Multithreading Acceleration

SUBP: Soft Uniform Block Pruning for 1 X N Sparse CNNs Multithreading Acceleration

Structured Probabilistic Pruning for Convolutional Neural Network Acceleration.

MaxQ: Multi-Axis Query for N:M Sparsity Network

Structured Pruning for Efficient Convolutional Neural Networks Via Incremental Regularization

A Convolutional Neural Network Accelerator Architecture with Fine-Granular Mixed Precision Configurability.

Exploring the Granularity of Sparsity in Convolutional Neural Networks.

Exploring the Regularity of Sparse Structure in Convolutional Neural Networks

Efficient Neural Networks with Spatial Wise Sparsity Using Unified Importance Map.

Learning Instance-wise Sparsity for Accelerating Deep Models

Exploring Fine-Grained Sparsity in Convolutional Neural Networks for Efficient Inference

Adaptive Structured Sparse Network for Efficient CNNs with Feature Regularization.

CRISP: Hybrid Structured Sparsity for Class-aware Model Pruning

Adaptive Pixel-wise Structured Sparse Network for Efficient CNNs

Intragroup Sparsity for Efficient Inference

Intermediate-grained kernel elements pruning with structured sparsity

SparseTrain: Exploiting Dataflow Sparsity for Efficient Convolutional Neural Networks Training