Abstract:Conventional artificial neural network (ANN) learning algorithms for classification tasks, either derivative-based optimization algorithms or derivative-free optimization algorithms work by training ANN first (or training and validating ANN) and then testing ANN, which are a two-stage and one-pass learning mechanism. Thus, this learning mechanism may not guarantee the generalization ability of a trained ANN. In this article, a novel bilevel learning model is constructed for self-organizing feed-forward neural network (FFNN), in which the training and testing processes are integrated into a unified framework. In this bilevel model, the upper level optimization problem is built for testing error on testing data set and network architecture based on network complexity, whereas the lower level optimization problem is constructed for network weights based on training error on training data set. For the bilevel framework, an interactive learning algorithm is proposed to optimize the architecture and weights of an FFNN with consideration of both training error and testing error. In this interactive learning algorithm, a hybrid binary particle swarm optimization (BPSO) taken as an upper level optimizer is used to self-organize network architecture, whereas the Levenberg–Marquardt (LM) algorithm as a lower level optimizer is utilized to optimize the connection weights of an FFNN. The bilevel learning model and algorithm have been tested on 20 benchmark classification problems. Experimental results demonstrate that the bilevel learning algorithm can significantly produce more compact FFNNs with more excellent generalization ability when compared with conventional learning algorithms.

Anefficient Learning Algorithm For Binary Feedforward Neural Networks

Extreme Learning Machine with Multiple Kernels

Sparse Bayesian Approach to Fast Learning Network for Multiclassification.

Dynamic node creation and fast learning algorithm for a hybrid feedforward neural network

A Batch Inheritance Extreme Learning Machine Algorithm Based on Regular Optimization

Extreme Learning Machine: A New Learning Scheme of Feedforward Neural Networks

A Bilevel Learning Model and Algorithm for Self-Organizing Feed-Forward Neural Networks for Pattern Classification.

A Novel Learning Algorithm of Single-Hidden-layer Feedforward Neural Networks

Evolving Feedforward Artificial Neural Networks Using a Two-Stage Approach

Communication-Efficient Federated Learning with Binary Neural Networks

An Improved Algorithm for Building Self-Organizing Feedforward Neural Networks

Optimal Algorithm for a Multilayer Feedforward Net and Its Application

A Fast Learning Algorithm for Feedfoward Neural Network Based on the Layer-by-Layer and Neuron-by-Neuron Optimizing Procedure

A Fast Learning Strategy For Multilayer Feedforward Neural Networks

Extreme learning machine: Theory and applications

Extreme Learning Machine With Adaptive Growth Of Hidden Nodes And Incremental Updating Of Output Weights

A Modified Algorithm for Feedforward Neural Networks

A L-BFGS Based Learning Algorithm for Complex-Valued Feedforward Neural Networks

A Mixed-Coding Adaptive Differential Evolution For Optimising The Architecture And Parameters Of Feedforward Neural Networks

A Conjugate Gradient-Based Efficient Algorithm for Training Single-Hidden-Layer Neural Networks.

Fast Learning Algorithms for Feedforward Neural Networks