Abstract:Deep artificial neural networks (DNNs) are typically trained via gradient-based learning algorithms, namely backpropagation. Evolution strategies (ES) can rival backprop-based algorithms such as Q-learning and policy gradients on challenging deep reinforcement learning (RL) problems. However, ES can be considered a gradient-based algorithm because it performs stochastic gradient descent via an operation similar to a finite-difference approximation of the gradient. That raises the question of whether non-gradient-based evolutionary algorithms can work at DNN scales. Here we demonstrate they can: we evolve the weights of a DNN with a simple, gradient-free, population-based genetic algorithm (GA) and it performs well on hard deep RL problems, including Atari and humanoid locomotion. The Deep GA successfully evolves networks with over four million free parameters, the largest neural networks ever evolved with a traditional evolutionary algorithm. These results (1) expand our sense of the scale at which GAs can operate, (2) suggest intriguingly that in some cases following the gradient is not the best choice for optimizing performance, and (3) make immediately available the multitude of neuroevolution techniques that improve performance. We demonstrate the latter by showing that combining DNNs with novelty search, which encourages exploration on tasks with deceptive or sparse reward functions, can solve a high-dimensional problem on which reward-maximizing algorithms (e.g.\ DQN, A3C, ES, and the GA) fail. Additionally, the Deep GA is faster than ES, A3C, and DQN (it can train Atari in ${\raise.17ex\hbox{$\scriptstyle\sim$}}$4 hours on one desktop or ${\raise.17ex\hbox{$\scriptstyle\sim$}}$1 hour distributed on 720 cores), and enables a state-of-the-art, up to 10,000-fold compact encoding technique.

GEN Model: An Alternative Approach to Deep Neural Network Models

Genetic Ensemble of Extreme Learning Machine

SEGEN: Sample-Ensemble Genetic Evolutional Network Model

GEGAN: Generative Adversarial Networks Training Based on Gradient Directed Evolutionary Strategy

Genetic-GNN: Evolutionary architecture search for Graph Neural Networks

GADAM: Genetic-Evolutionary ADAM for Deep Neural Network Optimization

Genetic-Gated Networks for Deep Reinforcement

Deep Neuroevolution of Recurrent and Discrete World Models

An Auto-Parallel Method for Deep Learning Models Based on Genetic Algorithm

Genetic Algorithms for Evolving Deep Neural Networks

Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning

Genetic Algorithm-based Optimization of Deep Neural Network Ensemble

Evolutionary Multiobjective Optimization Driven by Generative Adversarial Networks (Gans)

GenoHoption: Bridging Gene Network Graphs and Single-Cell Foundation Models

Learngene: From Open-World to Your Learning Task

Generative adversarial networks

BGADAM: Boosting based Genetic-Evolutionary ADAM for Neural Network Optimization

Neural Network Model for Genetic Algorithm

A NEW ALGORITHM OF EVOLVING ARTIFICIAL NEURAL NETWORKS VIA GENE EXPRESSION PROGRAMMING

Learning Generative Interactive Environments By Trained Agent Exploration

A Gradient-Guided Evolutionary Approach to Training Deep Neural Networks