Abstract:Machine learning approaches have recently been leveraged as a substitute or an aid for physical/mathematical modeling approaches to dynamical systems. To develop an efficient machine learning method dedicated to modeling and prediction of multiscale dynamics, we propose a reservoir computing (RC) model with diverse timescales by using a recurrent network of heterogeneous leaky integrator (LI) neurons. We evaluate computational performance of the proposed model in two time series prediction tasks related to four chaotic fast-slow dynamical systems. In a one-step-ahead prediction task where input data are provided only from the fast subsystem, we show that the proposed model yields better performance than the standard RC model with identical LI neurons. Our analysis reveals that the timescale required for producing each component of target multiscale dynamics is appropriately and flexibly selected from the reservoir dynamics by model training. In a long-term prediction task, we demonstrate that a closed-loop version of the proposed model can achieve longer-term predictions compared to the counterpart with identical LI neurons depending on the hyperparameter setting.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to develop a machine - learning model that can effectively simulate and predict the behavior of multi - scale dynamic systems. Specifically, the authors propose an Echo State Network with Diverse Time Scales (DTS - ESN) to address the challenges encountered by traditional methods when dealing with multi - scale dynamic systems. ### Problem Background Multi - scale dynamic systems are widely present in the fields of physics, biology, medicine, and engineering. These systems are composed of components at different time and space scales, and the complex interactions lead to multi - scale behaviors. In order to understand and predict these complex multi - scale systems, traditional physical/mathematical modeling methods face many difficulties, especially when the rules of the system (such as physical laws) are not fully known. Therefore, combining machine - learning methods has become a promising alternative or auxiliary tool. ### Research Objectives 1. **Develop a new machine - learning model**: By introducing diverse leak rates, construct an Echo State Network with Diverse Time Scales (DTS - ESN) to better adapt to the dynamic characteristics of different time scales. 2. **Improve prediction performance**: Evaluate the performance of the proposed DTS - ESN in two time - series prediction tasks, especially whether it can accurately predict the multi - scale behavior of the entire system when only the data of the fast subsystem is provided. 3. **Long - term prediction ability**: Verify the performance of DTS - ESN in long - term prediction, especially whether it can achieve more accurate long - term prediction in the closed - loop mode. ### Main Contributions - **Diverse time scales**: By introducing diverse leak - rate distributions, the model can be flexibly adjusted to the target dynamics of different time scales, thereby improving the adaptability and prediction accuracy of multi - scale dynamic systems. - **Flexibility and robustness**: Compared with the traditional single - leak - rate model, DTS - ESN can maintain high prediction performance in a wider range of parameters, especially when dealing with slow variables. - **Modular characteristics**: After the model is trained, the states of neurons are naturally separated into fast and slow subsystems, similar to the modular phenomenon in biological systems, which helps to understand the internal structure and function of complex systems. ### Conclusion The research shows that DTS - ESN exhibits significant advantages when dealing with multi - scale dynamic systems, especially in predicting slow variables and long - term prediction. This method not only improves prediction performance but also provides new ideas and tools for understanding and simulating complex multi - scale systems. Future research can further explore other types of heterogeneous components to enhance the generalization ability and application range of the model.

Reservoir Computing with Diverse Timescales for Prediction of Multiscale Dynamics

Hybridizing Traditional and Next-Generation Reservoir Computing to Accurately and Efficiently Forecast Dynamical Systems

An asynchronously deep reservoir computing for predicting chaotic time series

A systematic exploration of reservoir computing for forecasting complex spatiotemporal dynamics

Model-Free Prediction of Chaotic Systems Using High Efficient Next-generation Reservoir Computing

Oscillations enhance time-series prediction in reservoir computing with feedback

Model-Size Reduction for Reservoir Computing by Concatenating Internal States Through Time

Learning reservoir dynamics with temporal self-modulation

Prediction of Unobserved Bifurcation by Unsupervised Extraction of Slowly Time-Varying System Parameter Dynamics from Time Series Using Reservoir Computing

A Comparative Study of Reservoir Computing for Temporal Signal Processing

Predicting nonsmooth chaotic dynamics by reservoir computing

Higher-order Granger reservoir computing: simultaneously achieving scalable complex structures inference and accurate dynamics prediction

A Comparison Between Reservoir Computing with Recurrent Analogs in Predicting Dynamical Systems

Catch-22s of reservoir computing

A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning

A novel Reservoir Architecture for Periodic Time Series Prediction

Temporal Convolution Derived Multi-Layered Reservoir Computing

Backpropagation algorithms and Reservoir Computing in Recurrent Neural Networks for the forecasting of complex spatiotemporal dynamics

Predicting Chaotic System Behavior using Machine Learning Techniques

Enhancement of Multi-Timescale Reservoir Computing Based on Homogeneous Hardware Platform with Controllable Ionic Dynamics

Embedding Theory of Reservoir Computing and Reducing Reservoir Network Using Time Delays