Abstract:It is well-documented that cross-layer connections in feedforward small-world neural networks (FSWNNs) enhance the efficient transmission for gradients, thus improving its generalization ability with a fast learning. However, the merits of long-distance cross-layer connections are not fully utilized due to the random rewiring. In this study, aiming to further improve the learning efficiency, a fast FSWNN (FFSWNN) is proposed by taking into account the positive effects of long-distance cross-layer connections, and applied to nonlinear system modeling. First, a novel rewiring rule by giving priority to long-distance cross-layer connections is proposed to increase the gradient transmission efficiency when constructing FFSWNN. Second, an improved ridge regression method is put forward to determine the initial weights with high activation for the sigmoidal neurons in FFSWNN. Finally, to further improve the learning efficiency, an asynchronous learning algorithm is designed to train FFSWNN, with the weights connected to the output layer updated by the ridge regression method and other weights by the gradient descent method. Several experiments are conducted on four benchmark datasets from the University of California Irvine (UCI) machine learning repository and two datasets from real-life problems to evaluate the performance of FFSWNN on nonlinear system modeling. The results show that FFSWNN has significantly faster convergence speed and higher modeling accuracy than the comparative models, and the positive effects of the novel rewiring rule, the improved weight initialization, and the asynchronous learning algorithm on learning efficiency are demonstrated.

Extended Wang neural network for online solving a set of linear equations

RNN algorithm optimization based on extended unsaturated region

A Local Online Learning Approach for Non-linear Data.

A fast algorithm for solving large scale nonlinear optimization problems using RNN

Neural Network Solution for General Nonlinear Optimization Problems

A finite-time recurrent neural network for solving online time-varying Sylvester matrix equation based on a new evolution formula

Solving extended linear programming problems using a class of recurrent neural networks

Neural Network Approach for Solving Nonsingular Multi-Linear Tensor Systems

Nine novelties of ZNN differing from GNN with theorems and proofs for A(t)X(t)B(t)=C(t) solving as example

A New Neural Network Model And Its Application

Solving Differential Equation with Constrained Multilayer Feedforward Network.

Deep Neural Networks for Linear Sum Assignment Problems.

Robust gradient‐based neural networks for solving online the discrete periodic Lyapunov matrix equations

Predefined-time convergent neural networks for solving the time-varying nonsingular multi-linear tensor equations

A nonlinear zeroing neural network and its applications on time-varying linear matrix equations solving, electronic circuit currents computing and robotic manipulator trajectory tracking

New Zeroing Neural Network Models for Solving Nonstationary Sylvester Equation with Verifications on Mobile Manipulators.

A Fast Feedforward Small-World Neural Network for Nonlinear System Modeling

An Improved Finite Time Convergence Recurrent Neural Network with Application to Time-Varying Linear Complex Matrix Equation Solution

An Adaptive Gradient Neural Network to Solve Dynamic Linear Matrix Equations

An accelerated zeroing neural network for solving continuous coupled Lyapunov matrix equations

An efficient zeroing neural network for solving time-varying nonlinear equations