Abstract:Feedforward neural networks (FNNs) with a single hidden layer have been widely applied in data modeling due to its’ universal approximation capability to nonlinear maps. However, such a theoretical result does not provide with any guideline to determine the architecture of the model in practice. Thus, researches on self-organization of FNNs are useful and critical for effective data modeling. This paper proposes a hybrid constructing and pruning strategy (HCPS) for problem solving, where the mutual information (MI) and sensitivity analysis (SA) are employed to measure the amount of internal information of neurons at the hidden layer and the contribution rate of each hidden neuron, respectively. HCPS merges hidden neurons when their MI value becomes too high, deletes hidden neurons when their contribution rates are sufficiently small, and splits hidden neurons when their contribution rates are very big. For each instant pattern feed into the model as a training sample, the weights of the neural network will be updated to ensure the model's output unchanged during structural adjustment. HCPS aims to get a condensed model through eliminating redundant neurons and without degrading the instant modeling performance, which is associated with the model's generalization property. The proposed algorithm is evaluated by some benchmark data sets, including classification problems, a non-linear system identification problem, a time-series prediction problem, and a real world application for pM2.5 predictions. Simulation results with comparisons demonstrate that our proposed method performs favorably and has improved the existing work in terms of modeling performance.

Method to Determine Structure of Feedforward Neural Network by Optimization Calculation

A structure optimisation algorithm for feedforward neural network construction

An Optimization Algorithm on the Number of Hidden Layer Nodes in Feed-forward Neural Network

An Optimization Method for the Topological Structures of Feed-Forward Multi-Layer Neural Networks

Dynamic node creation and fast learning algorithm for a hybrid feedforward neural network

Optimal Algorithm for a Multilayer Feedforward Net and Its Application

Neural Network Structure Optimization for Hefei Light Source II Β Function Correction

Optimization Algorithm Inspired Deep Neural Network Structure Design

Variable Structure and Variable Learning Rate Fourier Neural Networks Research

An Improved Algorithm for Building Self-Organizing Feedforward Neural Networks

Structural Optimization of Feedforward Neural Network Based on Genetic Algorithms

Optimizing neuronal activation function types based on GP in constructive FNN design

Two-Input Power-Activation Neural Network Weights-Direct-Determination and Structure Optimized by Particle Swarm Optimization

Multiple Mittag-Leffler Stability of Almost Periodic Solutions for Fractional-Order Delayed Neural Networks: Distributed Optimization Approach

Research Development on Structure-Optimisation of Forward Neural Network

A novel Mathematical Modeling for Deep Multilayer Perceptron Optimization: Architecture Optimization and Activation Functions Selection

Efficient algorithm for the design of multilayer feedforward neural networks

Weights and Structure Determination of Multiple-Input Feed-Forward Neural Network Activated by Chebyshev Polynomials of Class 2 Via Cross-Validation

A Method for Estimating the Number of Hidden Neurons in Feed-Forward Neural Networks Based on Information Entropy

A Fast Learning Algorithm for Feedfoward Neural Network Based on the Layer-by-Layer and Neuron-by-Neuron Optimizing Procedure

Optimal Design of Neural Network Structure for Power System Frequency Security Constraints