Abstract:Recent advances in spiking neural networks (SNNs) have a predominant focus on network architectures, while relatively little attention has been paid to the underlying neuron model. The point neuron models, a cornerstone of deep SNNs, pose a bottleneck on the network-level expressivity since they depict somatic dynamics only. In contrast, the multi-compartment models in neuroscience offer remarkable expressivity by introducing dendritic morphology and dynamics, but remain underexplored in deep learning due to their unaffordable computational cost and inflexibility. To combine the advantages of both sides for a flexible, efficient yet more powerful model, we propose the dendritic spiking neuron (DendSN) incorporating multiple dendritic branches with nonlinear dynamics. Compared to the point spiking neurons, DendSN exhibits significantly higher expressivity. DendSN's flexibility enables its seamless integration into diverse deep SNN architectures. To accelerate dendritic SNNs (DendSNNs), we parallelize dendritic state updates across time steps, and develop Triton kernels for GPU-level acceleration. As a result, we can construct large-scale DendSNNs with depth comparable to their point SNN counterparts. Next, we comprehensively evaluate DendSNNs' performance on various demanding tasks. By modulating dendritic branch strengths using a context signal, catastrophic forgetting of DendSNNs is substantially mitigated. Moreover, DendSNNs demonstrate enhanced robustness against noise and adversarial attacks compared to point SNNs, and excel in few-shot learning settings. Our work firstly demonstrates the possibility of training bio-plausible dendritic SNNs with depths and scales comparable to traditional point SNNs, and reveals superior expressivity and robustness of reduced dendritic neuron models in deep learning, thereby offering a fresh perspective on advancing neural network design.

A lightweight data-driven spiking neuronal network model of Drosophila olfactory nervous system with dedicated hardware support

Odor Recognition with a Spiking Neural Network for Bioelectronic Nose

Deep CovDenseSNN: A Hierarchical Event-Driven Dynamic Framework with Spiking Neurons in Noisy Environment

Hierarchical Spiking-Based Model for Efficient Image Classification with Enhanced Feature Extraction and Encoding.

Mapping Very Large Scale Spiking Neuron Network to Neuromorphic Hardware.

CSNN: an Augmented Spiking Based Framework with Perceptron-Inception

Biologically Inspired Structure Learning with Reverse Knowledge Distillation for Spiking Neural Networks

A 5.37-TSOPS/W Reconfigurable Neuron Array with Dual-mode Neurons and Asynchronous Synapses for Energy-Efficient Inference and Biological Neural Network Simulation

Emergent Bio-Functional Similarities in a Cortical-Spike-Train-Decoding Spiking Neural Network Facilitate Predictions of Neural Computation

A neural algorithm for Drosophila linear and nonlinear decision-making

Unsupervised Spiking Neural Network with Dynamic Learning of Inhibitory Neurons

Application of a Brain-Inspired Spiking Neural Network Architecture to Odor Data Classification

Silicon Modeling of Spiking Neurons with Diverse Dynamic Behaviors

A Neuromorphic-Hardware Oriented Bio-Plausible Online-Learning Spiking Neural Network Model.

Flexible and Scalable Deep Dendritic Spiking Neural Networks with Multiple Nonlinear Branching

Modelling Spiking Neural Network from the Architecture Evaluation Perspective

ReplaceNet: real-time replacement of a biological neural circuit with a hardware-assisted spiking neural network

Efficient Modelling of Spiking Neural Networks on a Scalable Chip Multiprocessor

Automated customization of large-scale spiking network models to neuronal population activity

Accelerated Physical Emulation of Bayesian Inference in Spiking Neural Networks

A Simple Probabilistic Spiking Neuron Model with Hebbian Learning Rules.