Abstract:The brain's structure and functions have consistently inspired the development of intelligent technologies throughout human history.Brain-inspired spiking neural networks(SNNs)emulate neural dynamics by incorporating event-based computations to capture temporal information,thereby enabling brain-like and energy-efficient computations.Currently,SNNs have been successfully applied in various fields,such as image recognition and object detection,and have the potential to revolutionize the field of artificial intelligence.Similar to the brain,artificial models show emergent properties,such as high performance and intelligence,with sufficiently large size.However,the sizes of SNNs have been limited,which hinders potential performance improvements and thus their real-world application.Therefore,methods and theories for constructing large-scale SNNs must be developed to achieve their highest potential.New techniques for building large-scale SNNs have recently attracted extensive research interest.In this review,notable advances and developments with large-scale SNNs,including spiking neuron,algorithm and model structure designs,software frameworks and neuromorphic computing chips,and neuromorphic applications are summarized.Various strategies including parallel computing,surrogate gradients,and transformer-based SNNs are discussed in detail.Specifically,various types of spiking neurons have been developed,with parametric neurons improving the representation ability of SNNs and parallel spiking neurons accelerating the training process.The direct training method is notable for enabling the training of large-scale SNNs,while brain-inspired methods such as plasticity,attention,and pruning algorithms improve the robustness and energy efficiency of SNNs.The ResNet structure and its variants have led to the development of large-scale SNNs,and the emergence of transformer-based SNNs has significantly improved SNN performance on computer vision tasks.Programming frameworks have evolved from solely supporting the inference of large-scale networks to facilitating training,including automatic differentiation,parallel computation acceleration,and implementation on neuromorphic sensors and chips.Large SNNs are particularly well-suited for processing events/spikes collected by neuromorphic sensors,especially in image reconstruction and object detection,but their development is still in the early stages.Furthermore,existing challenges and deficiencies are discussed,providing guidance for the future development of large-scale SNN models.In summary,brain-inspired designs of models and neuromorphic chips are still limited to simplified simulations,lacking sufficient biological plausibility.The temporal dynamic representation capabilities of SNNs have not yet been fully demonstrated.The computations of large-scale SNNs must be accelerated while reducing their power consumption.Moreover,tasks such as natural language processing,reasoning,and decision-making and computer vision tasks must both be investigated.The problem of efficiently measuring and representing asynchronous spatiotemporal events/spike data remains unresolved,limiting the full potential of existing neuromorphic vision systems.In conclusion,large-scale brain-inspired neural networks can still be substantially improved.Efforts are needed in various research directions,including algorithms,computing frameworks,and architectures,to explore and optimize brain-like models and hardware in a collaborative manner.

Toward Large-scale Spiking Neural Networks: A Comprehensive Survey and Future Directions

Theories and Methods for Large-Scale Brain-Inspired Neural Networks

BitSNNs: Revisiting Energy-efficient Spiking Neural Networks

Spiking Deep Residual Networks.

Direct Training High-Performance Deep Spiking Neural Networks: A Review of Theories and Methods

Direct Learning-Based Deep Spiking Neural Networks: A Review

Towards Energy-Preserving Natural Language Understanding with Spiking Neural Networks

Toward Efficient Deep Spiking Neuron Networks:A Survey On Compression

Evolutionary Spiking Neural Networks: A Survey

SpikingMiniLM: Energy-Efficient Spiking Transformer for Natural Language Understanding

Scaling Spike-driven Transformer with Efficient Spike Firing Approximation Training

Adaptive Multi-Level Firing for Direct Training Deep Spiking Neural Networks

Towards High-performance Spiking Transformers from ANN to SNN Conversion

LDD: High-Precision Training of Deep Spiking Neural Network Transformers Guided by an Artificial Neural Network

Backpropagation-Based Learning Techniques for Deep Spiking Neural Networks: A Survey

Going Deeper With Directly-Trained Larger Spiking Neural Networks

Deep CovDenseSNN: A Hierarchical Event-Driven Dynamic Framework with Spiking Neurons in Noisy Environment

Direct Training for Spiking Neural Networks: Faster, Larger, Better

Spike Trains Encoding and Threshold Rescaling Method for Deep Spiking Neural Networks

A Review of Computing with Spiking Neural Networks

Toward Scalable, Efficient, and Accurate Deep Spiking Neural Networks With Backward Residual Connections, Stochastic Softmax, and Hybridization