Abstract:Feedforward neural networks (FNNs) with a single hidden layer have been widely applied in data modeling due to its’ universal approximation capability to nonlinear maps. However, such a theoretical result does not provide with any guideline to determine the architecture of the model in practice. Thus, researches on self-organization of FNNs are useful and critical for effective data modeling. This paper proposes a hybrid constructing and pruning strategy (HCPS) for problem solving, where the mutual information (MI) and sensitivity analysis (SA) are employed to measure the amount of internal information of neurons at the hidden layer and the contribution rate of each hidden neuron, respectively. HCPS merges hidden neurons when their MI value becomes too high, deletes hidden neurons when their contribution rates are sufficiently small, and splits hidden neurons when their contribution rates are very big. For each instant pattern feed into the model as a training sample, the weights of the neural network will be updated to ensure the model's output unchanged during structural adjustment. HCPS aims to get a condensed model through eliminating redundant neurons and without degrading the instant modeling performance, which is associated with the model's generalization property. The proposed algorithm is evaluated by some benchmark data sets, including classification problems, a non-linear system identification problem, a time-series prediction problem, and a real world application for pM2.5 predictions. Simulation results with comparisons demonstrate that our proposed method performs favorably and has improved the existing work in terms of modeling performance.

On Constructing a Hidden Layer for Three-Layered Feedforward Neural Networks

A Hidden Layer Evaluation Method for Feedforward Neural Networks Based on Subspace Analysis

Hidden Layer Performance Evaluation for Feedforward Neural Networks Based on Hidden Layer Growing Strategy

Comparative investigation on dimension reduction and regression in three layer feed-forward neural network

Lp Simultaneous Approximation by Neural Networks with One Hidden Layer

On the Principles of ReLU Networks with One Hidden Layer

A Multilayer Complex Neural Network Training Algorithm and Its Application in Adaptive Equalization

An Optimization Algorithm on the Number of Hidden Layer Nodes in Feed-forward Neural Network

Notes On The Hidden Nodes Of Multilayer Perceptron Classifiers

An Improved Algorithm for Building Self-Organizing Feedforward Neural Networks

On the Transformation Mechanisms of Multilayer Perceptrons with Sigmoid Activation Functions for Classifications

Neural Hilbert Ladders: Multi-Layer Neural Networks in Function Space

A structure optimisation algorithm for feedforward neural network construction

Multilayer neural networks with extensively many hidden units

A Layer-Wise Theoretical Framework for Deep Learning of Convolutional Neural Networks

Consideration on the learning efficiency of multiple-layered neural networks with linear units

Hidden Unit Specialization in Layered Neural Networks: ReLU vs. Sigmoidal Activation

Explaining the Behavior of Neuron Activations in Deep Neural Networks

Bkyy Three Layer Net Learning, Em-Like Algorithm, And Selection Criterion For Hidden Unit Number

An Optimization Method for the Topological Structures of Feed-Forward Multi-Layer Neural Networks

Adaptive two-layer ReLU neural network: I. Best least-squares approximation