Abstract:By exploiting discrete signal processing and simulating brain neuron communication, Spiking Neural Networks (SNNs) offer a low-energy alternative to Artificial Neural Networks (ANNs). However, existing SNN models, still face high computational costs due to the numerous time steps as well as network depth and scale. The tens of billions of neurons and trillions of synapses in the human brain are developed from only 20,000 genes, which inspires us to design an efficient genetic encoding strategy that dynamic evolves to regulate large-scale deep SNNs at low cost. Therefore, we first propose a genetically scaled SNN encoding scheme that incorporates globally shared genetic interactions to indirectly optimize neuronal encoding instead of weight, which obviously brings about reductions in parameters and energy consumption. Then, a spatio-temporal evolutionary framework is designed to optimize the inherently initial wiring rules. Two dynamic regularization operators in the fitness function evolve the neuronal encoding to a suitable distribution and enhance information quality of the genetic interaction respectively, substantially accelerating evolutionary speed and improving efficiency. Experiments show that our approach compresses parameters by approximately 50\% to 80\%, while outperforming models on the same architectures by 0.21\% to 4.38\% on CIFAR-10, CIFAR-100 and ImageNet. In summary, the consistent trends of the proposed genetically encoded spatio-temporal evolution across different datasets and architectures highlight its significant enhancements in terms of efficiency, broad scalability and robustness, demonstrating the advantages of the brain-inspired evolutionary genetic coding for SNN optimization.

High Dimensional Evolutionary Strategy for Tuning the Structure and Parameters of a Neural Network

Generative Evolutionary Strategy For Black-Box Optimizations

Genetic-GNN: Evolutionary architecture search for Graph Neural Networks

Optimizing Feed-Forward Neural Networks Using Cascaded Genetic Algorithm

A 32-year-old man with recurrent cholangitis

Genetic Algorithms and Evolutionary Programming Hybrid Strategy for Structure and Weight Learning for Multilayer Feedforward Neural Networks

Optimizing Deep Neural Networks with Multiple Search Neuroevolution

Evolving Deep Neural Networks via Cooperative Coevolution With Backpropagation

A Fast Evolutionary Learning to Optimize CNNInspec keywordsOther keywordsKey words

A Self-adaptive neuroevolution approach to constructing Deep Neural Network architectures across different types

A Fast Evolutionary Learning to Optimize CNN

Hybrid Optimization Strategy for Radial Basis Function Neural Network Structure

Sampled Training and Node Inheritance for Fast Evolutionary Neural Architecture Search

A structure optimisation algorithm for feedforward neural network construction

Genetic algorithm for neural network architecture optimization

An effective algorithm for hyperparameter optimization of neural networks

Neural Networks Designing Neural Networks: Multi-Objective Hyper-Parameter Optimization

A Gradient-Guided Evolutionary Approach to Training Deep Neural Networks

Evolving Efficient Genetic Encoding for Deep Spiking Neural Networks

NGDE: A Niching-Based Gradient-Directed Evolution Algorithm for Nonconvex Optimization

Designing neural networks using hybrid particle swarm optimization