Abstract:Deep spiking neural networks (SNNs) have emerged as a potential alternative to traditional deep learning frameworks, due to their promise to provide increased compute efficiency on event-driven neuromorphic hardware. However, to perform well on complex vision applications, most SNN training frameworks yield large inference latency which translates to increased spike activity and reduced energy efficiency. Hence,minimizing average spike activity while preserving accuracy indeep SNNs remains a significant challenge and <a class="link-external link-http" href="http://opportunity.This" rel="external noopener nofollow">this http URL</a> paper presents a non-iterative SNN training technique thatachieves ultra-high compression with reduced spiking activitywhile maintaining high inference accuracy. In particular, our framework first uses the attention-maps of an un compressed meta-model to yield compressed ANNs. This step can be tuned to support both irregular and structured channel pruning to leverage computational benefits over a broad range of platforms. The framework then performs sparse-learning-based supervised SNN training using direct inputs. During the training, it jointly optimizes the SNN weight, threshold, and leak parameters to drastically minimize the number of time steps required while retaining compression. To evaluate the merits of our approach, we performed experiments with variants of VGG and ResNet, on both CIFAR-10 and CIFAR-100, and VGG16 on <a class="link-external link-http" href="http://Tiny-ImageNet.The" rel="external noopener nofollow">this http URL</a> SNN models generated through the proposed technique yield SOTA compression ratios of up to 33.4x with no significant drops in accuracy compared to baseline unpruned counterparts. Compared to existing SNN pruning methods, we achieve up to 8.3x higher compression with improved accuracy.

A Soft-Pruning Method Applied During Training of Spiking Neural Networks for In-memory Computing Applications

Efficient Structure Slimming for Spiking Neural Networks

Class-Aware Pruning for Efficient Neural Networks

BitSNNs: Revisiting Energy-efficient Spiking Neural Networks

Towards Energy Efficient Spiking Neural Networks: an Unstructured Pruning Framework.

Brain-Inspired Efficient Pruning: Exploiting Criticality in Spiking Neural Networks

State Transition of Dendritic Spines Improves Learning of Sparse Spiking Neural Networks.

Neural Dynamics Pruning for Energy-Efficient Spiking Neural Networks

Adaptive Multi-Level Firing for Direct Training Deep Spiking Neural Networks

DynSNN: A Dynamic Approach to Reduce Redundancy in Spiking Neural Networks

FSpiNN: An Optimization Framework for Memory- and Energy-Efficient Spiking Neural Networks

Towards Low-Latency Energy-Efficient Deep SNNs via Attention-Guided Compression

An efficient pruning and fine-tuning method for deep spiking neural network

A A 22nm 0.43pJ/SOP Sparsity-Aware In-Memory Neuromorphic Computing System with Hybrid Spiking and Artificial Neural Network and Configurable Topology

Pruning of Deep Spiking Neural Networks Through Gradient Rewiring.

Efficient Training of Supervised Spiking Neural Network Via Accurate Synaptic-Efficiency Adjustment Method

You Only Spike Once: Improving Energy-Efficient Neuromorphic Inference to ANN-Level Accuracy

Reconsidering the energy efficiency of spiking neural networks

Towards Efficient Deep Spiking Neural Networks Construction with Spiking Activity based Pruning

Spatio-Temporal Pruning and Quantization for Low-latency Spiking Neural Networks

A Unified Evaluation Framework for Spiking Neural Network Hardware Accelerators Based on Emerging Non-Volatile Memory Devices