Abstract:It is well-documented that cross-layer connections in feedforward small-world neural networks (FSWNNs) enhance the efficient transmission for gradients, thus improving its generalization ability with a fast learning. However, the merits of long-distance cross-layer connections are not fully utilized due to the random rewiring. In this study, aiming to further improve the learning efficiency, a fast FSWNN (FFSWNN) is proposed by taking into account the positive effects of long-distance cross-layer connections, and applied to nonlinear system modeling. First, a novel rewiring rule by giving priority to long-distance cross-layer connections is proposed to increase the gradient transmission efficiency when constructing FFSWNN. Second, an improved ridge regression method is put forward to determine the initial weights with high activation for the sigmoidal neurons in FFSWNN. Finally, to further improve the learning efficiency, an asynchronous learning algorithm is designed to train FFSWNN, with the weights connected to the output layer updated by the ridge regression method and other weights by the gradient descent method. Several experiments are conducted on four benchmark datasets from the University of California Irvine (UCI) machine learning repository and two datasets from real-life problems to evaluate the performance of FFSWNN on nonlinear system modeling. The results show that FFSWNN has significantly faster convergence speed and higher modeling accuracy than the comparative models, and the positive effects of the novel rewiring rule, the improved weight initialization, and the asynchronous learning algorithm on learning efficiency are demonstrated.

Multilayer Feedforward Small-World Neural Networks and Its Function Approximation

A New Multilayer Feedforward Small-World Neural Network with Its Performances on Function Approximation

A New Small-World Neural Network with its Performance on Fault Tolerance

Analysis of Contrasting Neural Network with Small-World Network

A NOVEL SMALL-WORLD NETWORK MODEL: KEEPING CONNECTIVITY WITHOUT ADDING EDGES

A pinwheel-shaped deterministic small-world network

Approximation Capability of a Novel Neural Network Model for Dynamic Systems

Deterministic Small-World Network Extended from Triangles.

A Tree-Structured Deterministic Small-World Network

Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neurons

Random Vector Functional Link Networks for Function Approximation on Manifolds

Random Walks on Stochastic and Deterministic Small-World Networks.

Approximation by random networks with bounded number of layers

A Fast Feedforward Small-World Neural Network for Nonlinear System Modeling

Beyond Multilayer Perceptrons: Investigating Complex Topologies in Neural Networks

Stability of a neural network model with small-world connections

A Multilayer Feed Forward Small-World Neural Network Controller and Its Application on Electrohydraulic Actuation System.

Deep Stochastic Configuration Networks with Universal Approximation Property

Deterministic multidimensional growth model for small-world networks

Function Approximation with Randomly Initialized Neural Networks for Approximate Model Reference Adaptive Control

Why Deep Neural Networks for Function Approximation?