Abstract:By exploiting discrete signal processing and simulating brain neuron communication, Spiking Neural Networks (SNNs) offer a low-energy alternative to Artificial Neural Networks (ANNs). However, existing SNN models, still face high computational costs due to the numerous time steps as well as network depth and scale. The tens of billions of neurons and trillions of synapses in the human brain are developed from only 20,000 genes, which inspires us to design an efficient genetic encoding strategy that dynamic evolves to regulate large-scale deep SNNs at low cost. Therefore, we first propose a genetically scaled SNN encoding scheme that incorporates globally shared genetic interactions to indirectly optimize neuronal encoding instead of weight, which obviously brings about reductions in parameters and energy consumption. Then, a spatio-temporal evolutionary framework is designed to optimize the inherently initial wiring rules. Two dynamic regularization operators in the fitness function evolve the neuronal encoding to a suitable distribution and enhance information quality of the genetic interaction respectively, substantially accelerating evolutionary speed and improving efficiency. Experiments show that our approach compresses parameters by approximately 50\% to 80\%, while outperforming models on the same architectures by 0.21\% to 4.38\% on CIFAR-10, CIFAR-100 and ImageNet. In summary, the consistent trends of the proposed genetically encoded spatio-temporal evolution across different datasets and architectures highlight its significant enhancements in terms of efficiency, broad scalability and robustness, demonstrating the advantages of the brain-inspired evolutionary genetic coding for SNN optimization.

Accurate and Fast Deep Evolutionary Networks Structured Representation Through Activating and Freezing Dense Networks

Fast Deep Learning Training Through Intelligently Freezing Layers

FreezeOut: Accelerate Training by Progressively Freezing Layers

Evolutionary Shallowing Deep Neural Networks at Block Levels

Large-Scale Data-Driven Optimization in Deep Modeling With an Intelligent Decision-Making Mechanism

Fast-DENSER++: Evolving Fully-Trained Deep Artificial Neural Networks

Accelerating Deep Learning Inference via Freezing

Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science

Evolving Normalization-Activation Layers

Compression of deep neural networks: bridging the gap between conventional-based pruning and evolutionary approach

An Efficient Evolutionary Deep Learning Framework Based on Multi-source Transfer Learning to Evolve Deep Convolutional Neural Networks

A Fast Evolutionary Learning to Optimize CNNInspec keywordsOther keywordsKey words

Egeria: Efficient DNN Training with Knowledge-Guided Layer Freezing

Real-time Memory Efficient Large-pose Face Alignment via Deep Evolutionary Network

A Fast Evolutionary Learning to Optimize CNN

Evolving Efficient Genetic Encoding for Deep Spiking Neural Networks

Differential Evolution Based Layer-Wise Weight Pruning for Compressing Deep Neural Networks

An Efficient Convolutional Neural Network with Attached Accelerating Strategy

Efficient Network Construction Through Structural Plasticity

A Gradient-Guided Evolutionary Approach to Training Deep Neural Networks

DECN: Automated Evolutionary Algorithms Via Evolution Inspired Deep Convolution Network