Abstract:The inequality constraint of stochastic configuration networks (SCNs) is the key to guarantee its universal approximation capability. SCNs add new units to neural networks incrementally under its supervisory mechanism. However, with the increasing number of hidden nodes, the output weights which are evaluated by the least squares method are frequently unstable. This unstable phenomenon (arbitrarily small perturbations may lead to large changes in the solutions) can be classified as ill-posed problems. One of the most important properties of ill-posed problems is instability and the widely used Tikhonov regularization method to deal with ill-posed problems is undesirable. This paper studies the supervisory mechanism of SCNs and algebraic properties of the hidden output matrix. Then a new greedy stochastic configuration network (GSCN) for ill-posed problems is presented. Firstly, under the premise of ensuring the universal approximation capability, GSCN tries to optimize the randomly assigned input weights and biases based on hunter–prey optimization (HPO) algorithm. The optimized hidden parameters which can reduce the residual error most are selected for the newly added hidden node. Secondly, the singular value decomposition (SVD) and orthogonal-triangular (QR) decomposition with column pivoting are introduced to extract linearly independent subset of the hidden output matrix for ill-posed problems of SCNs. Finally, one function approximation, six benchmark regression, two classification datasets and bearing fault diagnosis are employed to evaluate the performance of GSCN. Experimental results indicate that GSCN shows superior performance with respect to fast convergence, generalization and stability power than other contrast methods. The code is available at https://github.com/Gzu-public-bigdata/GSCN .

Stochastic configuration broad learning system and its approximation capability analysis

Incremental Multi-Layer Broad Learning System with Stochastic Configuration Algorithm for Regression

Learning Probabilistic Models for Model Checking: an Evolutionary Approach and an Empirical Study

Universal Approximation Capability of Broad Learning System and Its Structural Variations

Stochastic Configuration Networks: Fundamentals and Algorithms

TKS-BLS: Temporal Kernel Stationary Broad Learning System for Enhanced Modeling, Anomaly Detection, and Incremental Learning with Application to Ironmaking Processes

Stochastic configuration networks with block increments for data modeling in process industries

Enhancing Deep Stochastic Configuration Networks: Efficient Training via Low-Rank Matrix Approximation

Kernel Stochastic Configuration Networks for Nonlinear Regression

Greedy stochastic configuration networks for ill-posed problems

Broad Minimax Probability Learning System and its Application in Regression Modeling

Adaptive inertia weights: an effective way to improve parameter estimation of hidden layer in stochastic configuration networks

Deeper Insights into Learning Performance of Stochastic Configuration Networks

Stochastic Successive Convex Approximation for Non-Convex Constrained Stochastic Optimization

A holistic global-local stochastic configuration network modeling framework with antinoise awareness for efficient semi-supervised regression

Stochastic configuration network for structural reliability analysis

Recurrent Broad Learning Systems for Time Series Prediction

Broad learning system: Feature extraction based on K-means clustering algorithm

Recurrent Stochastic Configuration Networks with Hybrid Regularization for Nonlinear Dynamics Modelling

Incremental Bayesian broad learning system and its industrial application

Analysis and Variants of Broad Learning System