Abstract:It is well-documented that cross-layer connections in feedforward small-world neural networks (FSWNNs) enhance the efficient transmission for gradients, thus improving its generalization ability with a fast learning. However, the merits of long-distance cross-layer connections are not fully utilized due to the random rewiring. In this study, aiming to further improve the learning efficiency, a fast FSWNN (FFSWNN) is proposed by taking into account the positive effects of long-distance cross-layer connections, and applied to nonlinear system modeling. First, a novel rewiring rule by giving priority to long-distance cross-layer connections is proposed to increase the gradient transmission efficiency when constructing FFSWNN. Second, an improved ridge regression method is put forward to determine the initial weights with high activation for the sigmoidal neurons in FFSWNN. Finally, to further improve the learning efficiency, an asynchronous learning algorithm is designed to train FFSWNN, with the weights connected to the output layer updated by the ridge regression method and other weights by the gradient descent method. Several experiments are conducted on four benchmark datasets from the University of California Irvine (UCI) machine learning repository and two datasets from real-life problems to evaluate the performance of FFSWNN on nonlinear system modeling. The results show that FFSWNN has significantly faster convergence speed and higher modeling accuracy than the comparative models, and the positive effects of the novel rewiring rule, the improved weight initialization, and the asynchronous learning algorithm on learning efficiency are demonstrated.

Study on the fast learning algorithm of single-layer neural networks

Sparse Bayesian Approach to Fast Learning Network for Multiclassification.

Study on Fast Intelligent Learning Algorithm of Neural Network(i)

Dynamic node creation and fast learning algorithm for a hybrid feedforward neural network

A Fast Learning Algorithm of Feedforward Neural Networks by Using Novel Error Functions

A Fast Learning Algorithm of Neural Networks by Changing Error Functions

Fast Learning Algorithms for Feedforward Neural Networks

Fast Sparse Deep Neural Networks: Theory and Performance Analysis

A Fast Learning Algorithm Of Neural Network For The Training And Recognition Of The Phonemes

FANNC: A Fast Adaptive Neural Network Classifier

Fast Learning of Graph Neural Networks with Guaranteed Generalizability: One-hidden-layer Case

Fast and Accurate Classification with a Multi-Spike Learning Algorithm for Spiking Neurons.

A Fast Feedforward Small-World Neural Network for Nonlinear System Modeling

An Improved Brain-Inspired Emotional Learning Algorithm for Fast Classification

New Neural Network Realization Algorithm for Neyman-Person Criterion

A Fast Hybrid Algorithm Of Global Optimization For Feedforward Neural Networks

Functional Network Method for the Identification of Nonlinear Systems

New Dynamical Optimal Learning for Linear Multilayer FNN

Learning Algorithm of Stage Control NBP Network

Speeding up the Training of Neural Networks with the One-Step Procedure

Convergence of a modified gradient-based learning algorithm with penalty for single-hidden-layer feed-forward networks