Abstract:Recent advances in spiking neural networks (SNNs) have a predominant focus on network architectures, while relatively little attention has been paid to the underlying neuron model. The point neuron models, a cornerstone of deep SNNs, pose a bottleneck on the network-level expressivity since they depict somatic dynamics only. In contrast, the multi-compartment models in neuroscience offer remarkable expressivity by introducing dendritic morphology and dynamics, but remain underexplored in deep learning due to their unaffordable computational cost and inflexibility. To combine the advantages of both sides for a flexible, efficient yet more powerful model, we propose the dendritic spiking neuron (DendSN) incorporating multiple dendritic branches with nonlinear dynamics. Compared to the point spiking neurons, DendSN exhibits significantly higher expressivity. DendSN's flexibility enables its seamless integration into diverse deep SNN architectures. To accelerate dendritic SNNs (DendSNNs), we parallelize dendritic state updates across time steps, and develop Triton kernels for GPU-level acceleration. As a result, we can construct large-scale DendSNNs with depth comparable to their point SNN counterparts. Next, we comprehensively evaluate DendSNNs' performance on various demanding tasks. By modulating dendritic branch strengths using a context signal, catastrophic forgetting of DendSNNs is substantially mitigated. Moreover, DendSNNs demonstrate enhanced robustness against noise and adversarial attacks compared to point SNNs, and excel in few-shot learning settings. Our work firstly demonstrates the possibility of training bio-plausible dendritic SNNs with depths and scales comparable to traditional point SNNs, and reveals superior expressivity and robustness of reduced dendritic neuron models in deep learning, thereby offering a fresh perspective on advancing neural network design.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is that the existing deep spiking neural networks (SNNs) overly simplify the neuron model in their design, especially ignoring the complex structure and dynamic characteristics of dendrites. Specifically: 1. **Expression Ability Bottleneck**: The existing point neuron models only describe the dynamics of the cell body while ignoring the morphology and dynamic characteristics of dendrites, which limits the expression ability at the network level. 2. **Lack of Biological Realism**: Although multi - compartment models can better simulate the complex behavior of real neurons, they are difficult to be applied to large - scale deep learning tasks due to high computational cost and poor flexibility. 3. **Catastrophic Forgetting in Continuous Learning**: In continuous learning tasks, the existing SNNs are prone to the phenomenon of catastrophic forgetting, that is, forgetting the knowledge of old tasks when learning new tasks. 4. **Insufficient Robustness to Adversarial Attacks and Noise**: The existing SNNs perform poorly in the face of adversarial attacks and noise. To solve these problems, the author proposes a new neuron model - **Dendritic Spiking Neuron (DendSN)**, which combines multiple dendritic branches with non - linear dynamics. By introducing dendritic computation, DendSN improves the expression ability of a single neuron and can be extended to large - scale deep SNN architectures while maintaining computational efficiency. In addition, the author also proposes the following improvement measures: - **Parallelization Algorithm**: To accelerate the update of dendritic states, the author develops Triton kernels to fully utilize the computing power of GPUs. - **Dendritic Branch Gating (DBG)**: A new continuous learning algorithm that effectively reduces the interference between different tasks and alleviates the problem of catastrophic forgetting by adjusting the strength of dendritic branches in different task contexts. - **Robustness Enhancement**: Experiments show that DendSNN exhibits stronger robustness in adversarial attacks and noisy environments, and also has better performance in few - shot learning scenarios. In conclusion, this paper aims to improve the expression ability and biological realism of deep SNNs by introducing dendritic computation and solve the challenges of existing SNNs in continuous learning, robustness, and large - scale applications.

Flexible and Scalable Deep Dendritic Spiking Neural Networks with Multiple Nonlinear Branching

Adaptive Multi-Level Firing for Direct Training Deep Spiking Neural Networks

Deep CovDenseSNN: A Hierarchical Event-Driven Dynamic Framework with Spiking Neurons in Noisy Environment

Spiking Deep Residual Networks.

HybridSNN: Combining Bio-Machine Strengths by Boosting Adaptive Spiking Neural Networks.

Deep Spiking Neural Network Using Spatio-temporal Backpropagation with Variable Resistance.

Spike-driven Multi-Scale Learning with Hybrid Mechanisms of Spiking Dendrites.

Biologically Inspired Structure Learning with Reverse Knowledge Distillation for Spiking Neural Networks

An Efficient Learning Algorithm for Direct Training Deep Spiking Neural Networks

State Transition of Dendritic Spines Improves Learning of Sparse Spiking Neural Networks.

Advancing Spiking Neural Networks Toward Deep Residual Learning

Multi-Level Firing with Spiking DS-ResNet: Enabling Better and Deeper Directly-Trained Spiking Neural Networks

SSTDP: Supervised Spike Timing Dependent Plasticity for Efficient Spiking Neural Network Training

Parallel Spiking Neurons with High Efficiency and Ability to Learn Long-term Dependencies

Comprehensive Online Training and Deployment for Spiking Neural Networks

Temporal dendritic heterogeneity incorporated with spiking neural networks for learning multi-timescale dynamics

A 5.37-TSOPS/W Reconfigurable Neuron Array with Dual-mode Neurons and Asynchronous Synapses for Energy-Efficient Inference and Biological Neural Network Simulation

Deep Spiking Neural Network with Neural Oscillation and Spike-Phase Information.

High-parallelism Inception-like Spiking Neural Networks for Unsupervised Feature Learning

NADOL: Neuromorphic Architecture for Spike-Driven Online Learning by Dendrites

Temporal-Coded Deep Spiking Neural Network with Easy Training and Robust Performance.