Abstract:In artificial neural network (ANN) learning, empirical risk can be expressed by training error, while structural risk can be expressed by network complexity. Learning from data is often seen as a tradeoff between these two risks. Additionally, balancing training error and validation error can overcome the overfitting problem to some extent. It is known that the network performance is also related to regularization term. In order to consider four factors, i.e. training error, validation error, network complexity and regularization term simultaneously in the training process of a single-hidden layer feedforword neural network (SLFN), a many-objective coevolutionary learning algorithm (MOCELA) integrated with extreme learning machine auto-encoder (ELMAE), called MOCELA-ELMAE is presented. In MOCELA, the non-dominated sorting genetic algorithm III (NSGA-III) is improved for handling the many-objective optimization model with hybrid variables, where binary coding is used for structure learning and real coding is utilized for representing input parameters, referring to all input weights and hidden biases of the AE network. Output parameters of AE, i.e. output weights are analytically calculated by the non-iterative learning rule. The network structure and connection parameters of SLFN are determined based on those of AE. A set of Pareto optimal solutions are eventually collected by the MOCELA-ELMAE, which represents multiple optimal SLFNs. To make the final decision, three best SLFNs with minimum validation errors are selected as the base classifiers for selective ensemble learning. Extensive experiments are implemented on the benchmark classification data sets of UCI machine learning repository, and obvious improvements have been observed when the proposed MOCELA-ELMAE is compared with the NSGA-III based on hybrid coding and completely non-iterative learning of SLFN respectively. The experimental results also illustrate that the MOCELA-ELMAE performs much better than other state-of-the-art learning algorithms on many data sets.

A Novel Deep Learning Approach: Stacked Evolutionary Auto-Encoder

Evolutionary Hierarchical Sparse Extreme Learning Autoencoder Network for Object Recognition

On Adaptive Learning Framework for Deep Weighted Sparse Autoencoder: A Multiobjective Evolutionary Algorithm

An Experimental Study on Hyper-parameter Optimization for Stacked Auto-Encoders

Evolving Deep Convolutional Variational Autoencoders for Image Classification

Gated Stacked Target-Related Autoencoder: A Novel Deep Feature Extraction and Layerwise Ensemble Method for Industrial Soft Sensor Application

Deep Network Based on Stacked Orthogonal Convex Incremental Elm Autoencoders

Feature-Aligned Stacked Autoencoder: A Novel Semisupervised Deep Learning Model for Pattern Classification of Industrial Faults

Building Feature Space of Extreme Learning Machine with Sparse Denoising Stacked-Autoencoder.

A Gradient-Guided Evolutionary Approach to Training Deep Neural Networks

Extreme Learning Machine Based on Evolutionary Multi-objective Optimization

DECN: Automated Evolutionary Algorithms Via Evolution Inspired Deep Convolution Network

Stacked Convolutional Sparse Auto-Encoders for Representation Learning

A new Stack Autoencoder: Neighbouring Sample Envelope Embedded Stack Autoencoder Ensemble Model

Denoising Auto-Encoders Toward Robust Unsupervised Feature Representation.

Training Stacked Denoising Autoencoders for Representation Learning

An Efficient Evolutionary Deep Learning Framework Based on Multi-source Transfer Learning to Evolve Deep Convolutional Neural Networks

Many-objective coevolutionary learning algorithm with extreme learning machine auto-encoder for ensemble classifier of feedforward neural networks

DECN: Evolution Inspired Deep Convolution Network for Black-box Optimization

Genetic Algorithms for Evolving Deep Neural Networks

Denoising Deep Extreme Learning Machine for Sparse Representation