Abstract:Stochastic Configuration Network (SCN) has a powerful capability for regression and classification analysis. Traditionally, it is quite challenging to correctly determine an appropriate architecture for a neural network so that the trained model can achieve excellent performance for both learning and generalization. Compared with the known randomized learning algorithms for single hidden layer feed-forward neural networks, such as Randomized Radial Basis Function (RBF) Networks and Random Vector Functional-link (RVFL), the SCN randomly assigns the input weights and biases of the hidden nodes in a supervisory mechanism. Since the parameters in the hidden layers are randomly generated in uniform distribution, hypothetically, there is optimal randomness. Heavy-tailed distribution has shown optimal randomness in an unknown environment for finding some targets. Therefore, in this research, the authors used heavy-tailed distributions to randomly initialize weights and biases to see if the new SCN models can achieve better performance than the original SCN. Heavy-tailed distributions, such as Lévy distribution, Cauchy distribution, and Weibull distribution, have been used. Since some mixed distributions show heavy-tailed properties, the mixed Gaussian and Laplace distributions were also studied in this research work. Experimental results showed improved performance for SCN with heavy-tailed distributions. For the regression model, SCN-Lévy, SCN-Mixture, SCN-Cauchy, and SCN-Weibull used less hidden nodes to achieve similar performance with SCN. For the classification model, SCN-Mixture, SCN-Lévy, and SCN-Cauchy have higher test accuracy of 91.5%, 91.7% and 92.4%, respectively. Both are higher than the test accuracy of the original SCN.

Hierarchical-Bayesian-Based Sparse Stochastic Configuration Networks for Construction of Prediction Intervals

Mixed-Distribution-Based Robust Stochastic Configuration Networks for Prediction Interval Construction

Broad Stochastic Configuration Network for Regression

Orthogonal Stochastic Configuration Networks with Adaptive Construction Parameter for Data Analytics

Ensemble Stochastic Configuration Networks for Estimating Prediction Intervals: A Simultaneous Robust Training Algorithm and Its Application

Bayesian Network Parameter Learning Using Constraint-Based Data Extension Method

Adaptive inertia weights: an effective way to improve parameter estimation of hidden layer in stochastic configuration networks

Learning Bayesian Networks by Constrained Bayesian Estimation

Deeper Insights into Learning Performance of Stochastic Configuration Networks

Parallel stochastic configuration networks for large-scale data regression

Deep Stochastic Configuration Networks with Different Random Sampling Strategies

Optimal Randomness for Stochastic Configuration Network (SCN) with Heavy-Tailed Distributions

Bayesian Approach to Learn Bayesian Networks Using Data and Constraints.

Deep Stochastic Configuration Networks with Universal Approximation Property

Intuitionistic Fuzzy Stochastic Configuration Networks for Solving Binary Classification Problems

Stochastic Configuration Networks: Fundamentals and Algorithms

Training Bayesian Neural Networks with Sparse Subspace Variational Inference

Enhancing Deep Stochastic Configuration Networks: Efficient Training via Low-Rank Matrix Approximation

A Further Study on the Inequality Constraints in Stochastic Configuration Networks.

A Novel Stochastic Configuration Network with Iterative Learning Using Privileged Information and Its Application

Robust Stochastic Configuration Networks with Kernel Density Estimation for Uncertain Data Regression