Abstract:Stochastic Configuration Networks (SCNs) are a class of randomized neural networks that integrate randomized algorithms within an incremental learning framework. A defining feature of SCNs is the supervisory mechanism, which adaptively adjusts the distribution to generate effective random basis functions, thereby enabling error-free learning. In this paper, we present a comprehensive analysis of the impact of the supervisory mechanism on the learning performance of SCNs. Our findings reveal that the current SCN framework evaluates the effectiveness of each random basis function in reducing residual errors using a lower bound on its error reduction potential, which constrains SCNs' overall learning efficiency. Specifically, SCNs may fail to consistently select the most effective random candidate as the new basis function during each training iteration. To overcome this problem, we propose a novel method for evaluating the hidden layer's output matrix, supported by a new supervisory mechanism that accurately assesses the error reduction potential of random basis functions without requiring the computation of the Moore-Penrose inverse of the output matrix. This approach enhances the selection of basis functions, reducing computational complexity and improving the overall scalability and learning capabilities of SCNs. We introduce a Recursive Moore-Penrose Inverse-SCN (RMPI-SCN) training scheme based on the new supervisory mechanism and demonstrate its effectiveness through simulations over some benchmark datasets. Experiments show that RMPI-SCN outperforms the conventional SCN in terms of learning capability, underscoring its potential to advance the SCN framework for large-scale data modeling applications.

Distributed Sparse Learning for Stochastic Configuration Networks Via Alternating Direction Method of Multipliers

Distributed Stochastic Configuration Networks with Cooperative Learning Paradigm

A Finite Time Discrete Distributed Learning Algorithm Using Stochastic Configuration Network

Deep Stochastic Configuration Networks with Different Random Sampling Strategies

Stochastic Configuration Networks: Fundamentals and Algorithms

Stochastic configuration networks with CPU-GPU implementation for large-scale data analytics

Distributed Learning of Predictive Structures from Multiple Tasks over Networks

Distributed Sparse Total Least-Squares over Networks

Distributed Jointly Sparse Multitask Learning over Networks

Stochastic Alternating Direction Method of Multipliers for Byzantine-Robust Distributed Learning

Distributed Sparse Recursive Least-Squares over Networks

Federated Stochastic Configuration Networks for Distributed Data Analytics

Stochastic Configuration Networks Ensemble with Heterogeneous Features for Large-Scale Data Analytics

A Lightweight Learning Method for Stochastic Configuration Networks Using Non-Inverse Solution

Distributed Self-Paced Learning in Alternating Direction Method of Multipliers

Lightweight Stochastic Configuration Networks

Orthogonal Stochastic Configuration Networks with Adaptive Construction Parameter for Data Analytics

Towards More Efficient Stochastic Decentralized Learning: Faster Convergence and Sparse Communication

Deeper Insights into Learning Performance of Stochastic Configuration Networks

Greedy Sparsity-Promoting Algorithms for Distributed Learning

Two Dimensional Stochastic Configuration Networks for Image Data Analytics