Abstract:Feedforward neural networks (FNNs) with a single hidden layer have been widely applied in data modeling due to its’ universal approximation capability to nonlinear maps. However, such a theoretical result does not provide with any guideline to determine the architecture of the model in practice. Thus, researches on self-organization of FNNs are useful and critical for effective data modeling. This paper proposes a hybrid constructing and pruning strategy (HCPS) for problem solving, where the mutual information (MI) and sensitivity analysis (SA) are employed to measure the amount of internal information of neurons at the hidden layer and the contribution rate of each hidden neuron, respectively. HCPS merges hidden neurons when their MI value becomes too high, deletes hidden neurons when their contribution rates are sufficiently small, and splits hidden neurons when their contribution rates are very big. For each instant pattern feed into the model as a training sample, the weights of the neural network will be updated to ensure the model's output unchanged during structural adjustment. HCPS aims to get a condensed model through eliminating redundant neurons and without degrading the instant modeling performance, which is associated with the model's generalization property. The proposed algorithm is evaluated by some benchmark data sets, including classification problems, a non-linear system identification problem, a time-series prediction problem, and a real world application for pM2.5 predictions. Simulation results with comparisons demonstrate that our proposed method performs favorably and has improved the existing work in terms of modeling performance.

A Hidden Layer Evaluation Method for Feedforward Neural Networks Based on Subspace Analysis

Hidden Layer Performance Evaluation for Feedforward Neural Networks Based on Hidden Layer Growing Strategy

On Constructing a Hidden Layer for Three-Layered Feedforward Neural Networks

Dynamic node creation and fast learning algorithm for a hybrid feedforward neural network

A Multilayer Complex Neural Network Training Algorithm and Its Application in Adaptive Equalization

A neuron-wise subspace correction method for the finite neuron method

Optimal Algorithm for a Multilayer Feedforward Net and Its Application

Research of New Learning Method of Feedforward Neural Network

A Method for Estimating the Number of Hidden Neurons in Feed-Forward Neural Networks Based on Information Entropy

An Optimization Algorithm on the Number of Hidden Layer Nodes in Feed-forward Neural Network

Comparative investigation on dimension reduction and regression in three layer feed-forward neural network

A Structure-Guided Gauss-Newton Method for Shallow ReLU Neural Network

An Interpretable Denoising Layer for Neural Networks Based on Reproducing Kernel Hilbert Space and Its Application in Machine Fault Diagnosis

Adaptive two-layer ReLU neural network: I. Best least-squares approximation

An Improved Algorithm for Building Self-Organizing Feedforward Neural Networks

Bkyy Three Layer Net Learning, Em-Like Algorithm, And Selection Criterion For Hidden Unit Number

A Fast Learning Algorithm for Feedfoward Neural Network Based on the Layer-by-Layer and Neuron-by-Neuron Optimizing Procedure

Subspace method based on neural networks for eigenvalue problems

Notes On The Hidden Nodes Of Multilayer Perceptron Classifiers

Error Feature Analysis of Multilayered Feedforward Neural Networks

Adaptive Neural Network Subspace Method for Solving Partial Differential Equations with High Accuracy