Abstract:Training neural networks by using conventional supervised backpropagation algorithms is a challenging task. This is due to significant limitations, such as the risk for local minimum stagnation in the loss landscape of neural networks. That may prevent the network from finding the global minimum of its loss function and therefore slow its convergence speed. Another challenge is the vanishing and exploding gradients that may happen when the gradients of the loss function of the model become either infinitesimally small or unmanageably large during the training. That also hinders the convergence of the neural models. On the other hand, the traditional gradient-based algorithms necessitate the pre-selection of learning parameters such as the learning rates, activation function, batch size, stopping criteria, and others. Recent research has shown the potential of evolutionary optimization algorithms to address most of those challenges in optimizing the overall performance of neural networks. In this research, we introduce and validate an evolutionary optimization framework to train multilayer perceptrons, which are simple feedforward neural networks. The suggested framework uses the recently proposed evolutionary cooperative optimization algorithm, namely, the dynamic group-based cooperative optimizer. The ability of this optimizer to solve a wide range of real optimization problems motivated our research group to benchmark its performance in training multilayer perceptron models. We validated the proposed optimization framework on a set of five datasets for engineering applications, and we compared its performance against the conventional backpropagation algorithm and other commonly used evolutionary optimization algorithms. The simulations showed the competitive performance of the proposed framework for most examined datasets in terms of overall performance and convergence. For three benchmarking datasets, the proposed framework provided increases of 2.7%, 4.83%, and 5.13% over the performance of the second best-performing optimizers, respectively.

Competitive co-evolution of multi-layer perceptron classifiers

Long-Term Progress and Behavior Complexification in Competitive Co-Evolution

Global Progress in Competitive Co-Evolution: a Systematic Comparison of Alternative Methods

Evolutionary optimization framework to train multilayer perceptrons for engineering applications

CompetEvo: Towards Morphological Evolution from Competition

A cooperative coevolution-based pittsburgh learning classifier system embedded with memetic feature selection

Adaptive Memetic Computing for Evolutionary Multiobjective Optimization

New Pathways in Coevolutionary Computation

Competitive Coevolutionary Multi-Agent Systems: The Application to Mapping and Scheduling Problems

Class Binarization to NeuroEvolution for Multiclass Classification

Evolutionary Reinforcement Learning via Cooperative Coevolution

An Evolutionary Many-Objective Algorithm Based on a Novel Tournament Selection Strategy

Advances in co-evolutionary algorithms

Competitive Co-evolution of Predator and Prey Sensory-Motor Systems

Dynamical transitions in the evolution of learning algorithms by selection

Evolutionary learning of selection hyper-heuristics for text classification

Evolutionary convolutional neural network for image classification based on multi-objective genetic programming with leader–follower mechanism

Competing leaders grey wolf optimizer and its application for training multi-layer perceptron classifier

On Cooperative Coevolution and Global Crossover

An Efficient Application of Neuroevolution for Competitive Multiagent Learning

Evolution with Opponent-Learning Awareness