Abstract:Recurrent stochastic configuration networks (RSCNs) are a class of randomized learner models that have shown promise in modelling nonlinear dynamics. In many fields, however, the data generated by industry systems often exhibits nonstationary characteristics, leading to the built model performing well on the training data but struggling with the newly arriving data. This paper aims at developing a self-organizing version of RSCNs, termed as SORSCNs, to enhance the continuous learning ability of the network for modelling nonstationary data. SORSCNs can autonomously adjust the network parameters and reservoir structure according to the data streams acquired in real-time. The output weights are updated online using the projection algorithm, while the network structure is dynamically adjusted in the light of the recurrent stochastic configuration algorithm and an improved sensitivity analysis. Comprehensive comparisons among the echo state network (ESN), online self-learning stochastic configuration network (OSL-SCN), self-organizing modular ESN (SOMESN), RSCN, and SORSCN are carried out. Experimental results clearly demonstrate that the proposed SORSCNs outperform other models with sound generalization, indicating great potential in modelling nonlinear systems with nonstationary dynamics.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: How to enhance the continuous learning ability of neural network models when dealing with non - stationary data. Specifically, many existing models perform well on training data, but it is difficult to maintain performance when facing newly arrived data, especially in the case where data generated in industrial systems often exhibit non - stationary characteristics. For this reason, the author proposes a Self - Organizing Recurrent Stochastic Configuration Networks (SORSCNs) to improve the ability to model non - stationary data. ### Specific background and challenges of the problem 1. **Challenges of non - stationary data**: - Data generated by industrial systems usually has nonlinear, uncertain, and multivariable evolution behaviors. - These data may change dynamically and be non - stationary, making it difficult to establish accurate prediction models. - Non - stationary data is characterized by its statistical properties (such as mean, variance, and autocorrelation) changing over time. 2. **Limitations of existing methods**: - Many existing models rely on the error back - propagation (BP) algorithm to train neural networks, which will lead to slow convergence speed, difficult structure determination, and sensitivity to initial weights and learning parameters. - Although some randomization methods (such as Random Vector Functional Link Network RVFLN, Stochastic Configuration Networks SCNs, etc.) perform well in terms of fast learning and computational simplicity, these methods still have deficiencies when dealing with non - stationary data. 3. **Goals and solutions**: - Propose a self - organizing version of the Recurrent Stochastic Configuration Networks (SORSCNs) to enhance the continuous learning ability of non - stationary data streams. - SORSCNs can independently adjust network parameters and reservoir structures and make dynamic adjustments according to the data streams obtained in real - time. - The output weights are updated online through the projection algorithm, and the network structure is dynamically adjusted based on the Recurrent Stochastic Configuration algorithm and improved sensitivity analysis. ### Main contributions 1. **Hybrid self - learning framework**: - Consider the characteristics of real - time data streams and dynamically adjust network parameters and structures. - Adopt a unique growth - pruning - regrowth scheme to enhance the model's continuous learning ability for unknown non - stationary dynamics. 2. **Inherit the properties of the original RSCN**: - Naturally inherit the universal approximation property and echo state property of the original RSCN. 3. **Verification in practical applications**: - Applied to the predictive analysis of two real - world industrial data, the results show that the proposed SORSCNs are superior in terms of prediction accuracy and structural compactness. Through these improvements, the paper aims to solve the key challenges in non - stationary data modeling and provide a more effective prediction tool for complex industrial processes.

Self-Organizing Recurrent Stochastic Configuration Networks for Nonstationary Data Modelling

Recurrent Stochastic Configuration Networks for Temporal Data Analytics

Recurrent Stochastic Configuration Networks with Hybrid Regularization for Nonlinear Dynamics Modelling

Deep Recurrent Stochastic Configuration Networks for Modelling Nonlinear Dynamic Systems

RSNN: Recurrent Spiking Neural Networks for Dynamic Spatial-Temporal Information Processing

Relational State-Space Model for Stochastic Multi-Object Systems

Design of a self-organizing reciprocal modular neural network for nonlinear system modeling

Stochastic Configuration Networks: Fundamentals and Algorithms

Deeper Insights into Learning Performance of Stochastic Configuration Networks

Fuzzy Recurrent Stochastic Configuration Networks for Industrial Data Analytics

Kernel Stochastic Configuration Networks for Nonlinear Regression

A holistic global-local stochastic configuration network modeling framework with antinoise awareness for efficient semi-supervised regression

Nonlinear Dynamics Analysis of a Self-Organizing Recurrent Neural Network: Chaos Waning

Multi-grain Cascade Recurrent Neural Network for Nonlinear Time-varying Process Soft Sensor Modeling

Two Dimensional Stochastic Configuration Networks for Image Data Analytics

Probabilistic Sequential Network for Deep Learning of Complex Process Data and Soft Sensor Application

Stochastic configuration networks with block increments for data modeling in process industries

Hybrid Parallel Stochastic Configuration Networks for Industrial Data Analytics

Adaptive Reorganization of Neural Pathways for Continual Learning with Spiking Neural Networks

Neural Structure Learning with Stochastic Differential Equations

Chain-Structure Echo State Network With Stochastic Optimization: Methodology and Application