Abstract:The brain's structure and functions have consistently inspired the development of intelligent technologies throughout human history.Brain-inspired spiking neural networks(SNNs)emulate neural dynamics by incorporating event-based computations to capture temporal information,thereby enabling brain-like and energy-efficient computations.Currently,SNNs have been successfully applied in various fields,such as image recognition and object detection,and have the potential to revolutionize the field of artificial intelligence.Similar to the brain,artificial models show emergent properties,such as high performance and intelligence,with sufficiently large size.However,the sizes of SNNs have been limited,which hinders potential performance improvements and thus their real-world application.Therefore,methods and theories for constructing large-scale SNNs must be developed to achieve their highest potential.New techniques for building large-scale SNNs have recently attracted extensive research interest.In this review,notable advances and developments with large-scale SNNs,including spiking neuron,algorithm and model structure designs,software frameworks and neuromorphic computing chips,and neuromorphic applications are summarized.Various strategies including parallel computing,surrogate gradients,and transformer-based SNNs are discussed in detail.Specifically,various types of spiking neurons have been developed,with parametric neurons improving the representation ability of SNNs and parallel spiking neurons accelerating the training process.The direct training method is notable for enabling the training of large-scale SNNs,while brain-inspired methods such as plasticity,attention,and pruning algorithms improve the robustness and energy efficiency of SNNs.The ResNet structure and its variants have led to the development of large-scale SNNs,and the emergence of transformer-based SNNs has significantly improved SNN performance on computer vision tasks.Programming frameworks have evolved from solely supporting the inference of large-scale networks to facilitating training,including automatic differentiation,parallel computation acceleration,and implementation on neuromorphic sensors and chips.Large SNNs are particularly well-suited for processing events/spikes collected by neuromorphic sensors,especially in image reconstruction and object detection,but their development is still in the early stages.Furthermore,existing challenges and deficiencies are discussed,providing guidance for the future development of large-scale SNN models.In summary,brain-inspired designs of models and neuromorphic chips are still limited to simplified simulations,lacking sufficient biological plausibility.The temporal dynamic representation capabilities of SNNs have not yet been fully demonstrated.The computations of large-scale SNNs must be accelerated while reducing their power consumption.Moreover,tasks such as natural language processing,reasoning,and decision-making and computer vision tasks must both be investigated.The problem of efficiently measuring and representing asynchronous spatiotemporal events/spike data remains unresolved,limiting the full potential of existing neuromorphic vision systems.In conclusion,large-scale brain-inspired neural networks can still be substantially improved.Efforts are needed in various research directions,including algorithms,computing frameworks,and architectures,to explore and optimize brain-like models and hardware in a collaborative manner.

No One-Size-Fits-All Neurons: Task-based Neurons for Artificial Neural Networks

Mapping Very Large Scale Spiking Neuron Network to Neuromorphic Hardware.

Towards NeuroAI: Introducing Neuronal Diversity into Artificial Neural Networks

Neural networks with optimized single-neuron adaptation uncover biologically plausible regularization

IC Neuron: an Efficient Unit to Construct Neural Networks

A 5.37-TSOPS/W Reconfigurable Neuron Array with Dual-mode Neurons and Asynchronous Synapses for Energy-Efficient Inference and Biological Neural Network Simulation

Training Spiking Neural Networks for Cognitive Tasks: A Versatile Framework Compatible with Various Temporal Codes.

Brain-inspired Evolutionary Architectures for Spiking Neural Networks

Exploration Of A Mechanism To Form Bionic, Self-Growing And Self-Organizing Neural Network

Biologically Inspired Heterogeneous Learning for Accurate, Efficient and Low-Latency Neural Network

Modular Building Blocks for Mapping Spiking Neural Networks Onto a Programmable Neuromorphic Processor

Dendritic Neuron Model with Effective Learning Algorithms for Classification, Approximation, and Prediction

Contrastive Analysis of Neuron Model

What should a neuron aim for? Designing local objective functions based on information theory

Spiking Neural Networks Based Cortex Like Mechanism: A Case Study for Facial Expression Recognition

Neuralizer: General Neuroimage Analysis without Re-Training

Enhancing learning in spiking neural networks through neuronal heterogeneity and neuromodulatory signaling

Towards Brain Inspired Design for Addressing the Shortcomings of ANNs

Continual Learning with Deep Artificial Neurons

SIT: A Bionic and Non-Linear Neuron for Spiking Neural Network

Theories and Methods for Large-Scale Brain-Inspired Neural Networks