Abstract:When deploying deep neural networks (DNNs), edge devices training is practical to improve model adaptivity for various user-specific scenarios while avoiding privacy disclosure. However, the training computation is intolerable for edge devices. It inspires sparse DNN training (SDT) into the limelight, which reduces training computation by dynamic weight pruning. Generally, SDT has two strategies based on the pruning granularity: the structured or the unstructured. Unfortunately, both of them suffer from limited training efficiency due to the gap between pruning granularity and hardware implementation. The former is hardware-friendly but has a low pruning ratio, indicating limited computation reduction. The latter has a high pruning ratio, but the unbalanced workload decreases utilization and irregular sparsity distribution causes considerable sparsity processing overhead. This paper proposes a software-hardware co- design to bridge the gap for improving the efficiency of SDT. On the algorithm side, a sub-structured pruning method, achieved with hybrid shape-wise and line-wise pruning, generates a high sparsity ratio and keeps the hardware-friendly property. On the hardware side, a sub-structured weight processing unit (SWPU) effectively handles the hybrid sparsity with three techniques. First, SWPU dynamically reorders the computation sequence with hamming-distance-based clustering, balancing the irregular workload. Second, SWPU performs runtime scheduling by exploiting the feature of sub-structured sparse convolution through a detect-before-load controller, which skips redundant memory access and sparsity processing. Third, SWPU performs sparse convolution by compressing operands with spatial disconnect log-based routing and recovers their location with bi-directional switching, avoiding the power-consumed routing logic. Synthesized with 28nm CMOS technology, SWPU can enable 0.56V-to-1.0V supply voltage with a maximum frequency of 675 MHz. It achieves a 50.1% higher pruning ratio than structured pruning and $1.53\times $ higher energy efficiency than unstructured pruning. The peak energy efficiency of SWPU is 126.04TFLOPS/W, outperforming the state-of-the-art training processor by $1.67\times $ . When training a ResNet-18 model, SWPU reduces $3.72\times $ energy and offers $4.69\times $ speedup than previous sparse training processors.

A 400mhz NPU with 7.8TOPS2/W High-PerformanceGuaranteed Efficiency in 55nm for Multi-Mode Pruning and Diverse Quantization Using Pattern-Kernel Encoding and Reconfigurable MAC Units

Single-shot Pruning and Quantization for Hardware-Friendly Neural Network Acceleration

Class-Aware Pruning for Efficient Neural Networks

Harmonious Coexistence of Structured Weight Pruning and Ternarization for Deep Neural Networks

A fine-grained mixed precision DNN accelerator using a two-stage big-little core RISC-V MCU.

Differentiable Joint Pruning and Quantization for Hardware Efficiency

Joint Pruning and Channel-Wise Mixed-Precision Quantization for Efficient Deep Neural Networks

Pruning and quantization for deep neural network acceleration: A survey

SWPU: A 126.04 TFLOPS/W Edge-Device Sparse DNN Training Processor with Dynamic Sub-Structured Weight Pruning

Progressive DNN Compression: A Key to Achieve Ultra-High Weight Pruning and Quantization Rates using ADMM

PCNN: Pattern-based Fine-Grained Regular Pruning Towards Optimizing CNN Accelerators

HFPQ: Deep Neural Network Compression by Hardware-Friendly Pruning-Quantization

PowerPruning: Selecting Weights and Activations for Power-Efficient Neural Network Acceleration

A Dynamic Pruning Method on Multiple Sparse Structures in Deep Neural Networks

BitXpro: Regularity-Aware Hardware Runtime Pruning for Deep Neural Networks

Quantisation and Pruning for Neural Network Compression and Regularisation

PCONV: the Missing but Desirable Sparsity in DNN Weight Pruning for Real-Time Execution on Mobile Devices.

Efficient Micro-Structured Weight Unification and Pruning for Neural Network Compression

Exploiting Weight-Level Sparsity in Channel Pruning with Low-Rank Approximation

Structured Term Pruning for Computational Efficient Neural Networks Inference