Abstract:Recent advances in spiking neural networks (SNNs) have a predominant focus on network architectures, while relatively little attention has been paid to the underlying neuron model. The point neuron models, a cornerstone of deep SNNs, pose a bottleneck on the network-level expressivity since they depict somatic dynamics only. In contrast, the multi-compartment models in neuroscience offer remarkable expressivity by introducing dendritic morphology and dynamics, but remain underexplored in deep learning due to their unaffordable computational cost and inflexibility. To combine the advantages of both sides for a flexible, efficient yet more powerful model, we propose the dendritic spiking neuron (DendSN) incorporating multiple dendritic branches with nonlinear dynamics. Compared to the point spiking neurons, DendSN exhibits significantly higher expressivity. DendSN's flexibility enables its seamless integration into diverse deep SNN architectures. To accelerate dendritic SNNs (DendSNNs), we parallelize dendritic state updates across time steps, and develop Triton kernels for GPU-level acceleration. As a result, we can construct large-scale DendSNNs with depth comparable to their point SNN counterparts. Next, we comprehensively evaluate DendSNNs' performance on various demanding tasks. By modulating dendritic branch strengths using a context signal, catastrophic forgetting of DendSNNs is substantially mitigated. Moreover, DendSNNs demonstrate enhanced robustness against noise and adversarial attacks compared to point SNNs, and excel in few-shot learning settings. Our work firstly demonstrates the possibility of training bio-plausible dendritic SNNs with depths and scales comparable to traditional point SNNs, and reveals superior expressivity and robustness of reduced dendritic neuron models in deep learning, thereby offering a fresh perspective on advancing neural network design.

Deep Learning Depends on Segregated Dendritic Compartments and Spike Frequency Adaptation

Towards deep learning with segregated dendrites

Adaptive Multi-Level Firing for Direct Training Deep Spiking Neural Networks

Beyond spiking networks: The computational advantages of dendritic amplification and input segregation

Toward Efficient Processing and Learning with Spikes: New Approaches for Multispike Learning

Flexible and Scalable Deep Dendritic Spiking Neural Networks with Multiple Nonlinear Branching

Deep Spiking Neural Network Using Spatio-temporal Backpropagation with Variable Resistance.

IM-LIF: Improved Neuronal Dynamics with Attention Mechanism for Direct Training Deep Spiking Neural Network

Deep CovDenseSNN: A Hierarchical Event-Driven Dynamic Framework with Spiking Neurons in Noisy Environment

An Efficient Learning Algorithm for Direct Training Deep Spiking Neural Networks

Efficient Spike-Driven Learning With Dendritic Event-Based Processing

A Brain-Inspired Spiking Neural Network Model with Temporal Encoding and Learning

Dendritic cortical microcircuits approximate the backpropagation algorithm

Enhancing learning in spiking neural networks through neuronal heterogeneity and neuromodulatory signaling

Two-compartment neuronal spiking model expressing brain-state specific apical-amplification, -isolation and -drive regimes

Temporal dendritic heterogeneity incorporated with spiking neural networks for learning multi-timescale dynamics

Towards Biologically Plausible Deep Learning

Emergent Bio-Functional Similarities in a Cortical-Spike-Train-Decoding Spiking Neural Network Facilitate Predictions of Neural Computation

Multi-Level Firing with Spiking DS-ResNet: Enabling Better and Deeper Directly-Trained Spiking Neural Networks

TC-LIF: A Two-Compartment Spiking Neuron Model for Long-Term Sequential Modelling

Learning the Plasticity: Plasticity-Driven Learning Framework in Spiking Neural Networks