Abstract:In artificial neural network (ANN) learning, empirical risk can be expressed by training error, while structural risk can be expressed by network complexity. Learning from data is often seen as a tradeoff between these two risks. Additionally, balancing training error and validation error can overcome the overfitting problem to some extent. It is known that the network performance is also related to regularization term. In order to consider four factors, i.e. training error, validation error, network complexity and regularization term simultaneously in the training process of a single-hidden layer feedforword neural network (SLFN), a many-objective coevolutionary learning algorithm (MOCELA) integrated with extreme learning machine auto-encoder (ELMAE), called MOCELA-ELMAE is presented. In MOCELA, the non-dominated sorting genetic algorithm III (NSGA-III) is improved for handling the many-objective optimization model with hybrid variables, where binary coding is used for structure learning and real coding is utilized for representing input parameters, referring to all input weights and hidden biases of the AE network. Output parameters of AE, i.e. output weights are analytically calculated by the non-iterative learning rule. The network structure and connection parameters of SLFN are determined based on those of AE. A set of Pareto optimal solutions are eventually collected by the MOCELA-ELMAE, which represents multiple optimal SLFNs. To make the final decision, three best SLFNs with minimum validation errors are selected as the base classifiers for selective ensemble learning. Extensive experiments are implemented on the benchmark classification data sets of UCI machine learning repository, and obvious improvements have been observed when the proposed MOCELA-ELMAE is compared with the NSGA-III based on hybrid coding and completely non-iterative learning of SLFN respectively. The experimental results also illustrate that the MOCELA-ELMAE performs much better than other state-of-the-art learning algorithms on many data sets.

Genetic Algorithms and Evolutionary Programming Hybrid Strategy for Structure and Weight Learning for Multilayer Feedforward Neural Networks

Evolving Neural Networks Using the Hybrid of Ant Colony Optimization and Bp Algorithms

Dynamic node creation and fast learning algorithm for a hybrid feedforward neural network

Optimal Design of Composite Grid/skin Structures Based on Deep Learning and Double-Double Layup Strategy

Two Hybrid Learning Strategies of Feedforward Network

Optimizing Feed-Forward Neural Networks Using Cascaded Genetic Algorithm

Hybrid Optimization Strategy for Radial Basis Function Neural Network Structure

Genetic Algorithms for MLP Neural Network parameters optimization

A Hybrid Genetic Algorithm-Neural Network Strategy for Simulation Optimization.

An Improved Genetic Algorithm for Training Layered Feedforward Neural Networks

Multiple-scale Uncertainty Optimization Design of Hybrid Composite Structures Based on Neural Network and Genetic Algorithm

Novel hybrid evolutionary algorithm for multi-objective optimization problem

Genetic algorithm for neural network architecture optimization

An Evolutionary Algorithm Based on Fully Connected Weight Networks for Mixed-Variable Multi-objective Optimization

Many-objective coevolutionary learning algorithm with extreme learning machine auto-encoder for ensemble classifier of feedforward neural networks

Multistrategy Learning Using Genetic Algorithms and Neural Networks for Pattern Classification

An Evolutionary Algorithm Based on Fully Connected Weight Networks for Mixed-Variable Multiobjective Optimization

Designing neural networks using hybrid particle swarm optimization

Learning to Accelerate Evolutionary Search for Large-Scale Multiobjective Optimization

A hybrid training algorithm based on gradient descent and evolutionary computation

Personalized Hybrid Education Framework Based on Neuroevolution Methodologies