Abstract:Organizations managing high-performance computing systems face a multitude of challenges, including overarching concerns such as overall energy consumption, microprocessor clock frequency limitations, and the escalating costs associated with chip production. Evidently, processor speeds have plateaued over the last decade, persisting within the range of 2 GHz to 5 GHz. Scholars assert that brain-inspired computing holds substantial promise for mitigating these challenges. The spiking neural network (SNN) particularly stands out for its commendable power efficiency when juxtaposed with conventional design paradigms. Nevertheless, our scrutiny has brought to light several pivotal challenges impeding the seamless implementation of large-scale neural networks (NNs) on silicon. These challenges encompass the absence of automated tools, the need for multifaceted domain expertise, and the inadequacy of existing algorithms to efficiently partition and place extensive SNN computations onto hardware infrastructure. In this paper, we posit the development of an automated tool flow capable of transmuting any NN into an SNN. This undertaking involves the creation of a novel graph-partitioning algorithm designed to strategically place SNNs on a network-on-chip (NoC), thereby paving the way for future energy-efficient and high-performance computing paradigms. The presented methodology showcases its effectiveness by successfully transforming ANN architectures into SNNs with a marginal average error penalty of merely 2.65%. The proposed graph-partitioning algorithm enables a 14.22% decrease in inter-synaptic communication and an 87.58% reduction in intra-synaptic communication, on average, underscoring the effectiveness of the proposed algorithm in optimizing NN communication pathways. Compared to a baseline graph-partitioning algorithm, the proposed approach exhibits an average decrease of 79.74% in latency and a 14.67% reduction in energy consumption. Using existing NoC tools, the energy-latency product of SNN architectures is, on average, 82.71% lower than that of the baseline architectures.

Enabling Energy-Efficient and Reliable Neural Network via Neuron-Level Voltage Scaling

BitSNNs: Revisiting Energy-efficient Spiking Neural Networks

Scaling Graph Neural Networks for Large-Scale Power Systems Analysis: Empirical Laws for Emergent Abilities

A Quality-Aware Voltage Overscaling Framework to Improve the Energy Efficiency and Lifetime of TPUs based on Statistical Error Modeling

Efficient Structure Slimming for Spiking Neural Networks

Optimizing Power Efficiency: Converting Recurrent Neural Networks to Spiking Neural Networks for Time-Domain Analysis

Spike Trains Encoding and Threshold Rescaling Method for Deep Spiking Neural Networks

Towards Energy-Preserving Natural Language Understanding with Spiking Neural Networks

NeuralFuse: Learning to Recover the Accuracy of Access-Limited Neural Network Inference in Low-Voltage Regimes

Convergence Enhancement for Neural Network Integrated Power System Time Domain Simulation

Scaling Spike-driven Transformer with Efficient Spike Firing Approximation Training

Recent Advances in Scalable Energy-Efficient and Trustworthy Spiking Neural networks: from Algorithms to Technology

Shavette: Low Power Neural Network Acceleration via Algorithm-level Error Detection and Undervolting

Towards High-performance Spiking Transformers from ANN to SNN Conversion

Neural Networks Battery Applications: A Review

NeuralScale: Efficient Scaling of Neurons for Resource-Constrained Deep Neural Networks

Toward High-Accuracy and Low-Latency Spiking Neural Networks With Two-Stage Optimization

EnforceSNN: Enabling Resilient and Energy-Efficient Spiking Neural Network Inference considering Approximate DRAMs for Embedded Systems

Adaptive Synaptic Scaling in Spiking Networks for Continual Learning and Enhanced Robustness

Improving The Scalability Of Deep Neural Network For Probabilistic Power Flow

Benchmarking Artificial Neural Network Architectures for High-Performance Spiking Neural Networks