Abstract:Recent work has shown that machine learning (ML) models can skillfully forecast the dynamics of unknown chaotic systems. Short-term predictions of the state evolution and long-term predictions of the statistical patterns of the dynamics ("climate") can be produced by employing a feedback loop, whereby the model is trained to predict forward only one time step, then the model output is used as input for multiple time steps. In the absence of mitigating techniques, however, this feedback can result in artificially rapid error growth ("instability"). One established mitigating technique is to add noise to the ML model training input. Based on this technique, we formulate a new penalty term in the loss function for ML models with memory of past inputs that deterministically approximates the effect of many small, independent noise realizations added to the model input during training. We refer to this penalty and the resulting regularization as Linearized Multi-Noise Training (LMNT). We systematically examine the effect of LMNT, input noise, and other established regularization techniques in a case study using reservoir computing, a machine learning method using recurrent neural networks, to predict the spatiotemporal chaotic Kuramoto-Sivashinsky equation. We find that reservoir computers trained with noise or with LMNT produce climate predictions that appear to be indefinitely stable and have a climate very similar to the true system, while the short-term forecasts are substantially more accurate than those trained with other regularization techniques. Finally, we show the deterministic aspect of our LMNT regularization facilitates fast reservoir computer regularization hyperparameter tuning.

Long-term Prediction of Chaotic Systems with Machine Learning

Long-term prediction of chaotic systems with recurrent neural networks

Predicting phase and sensing phase coherence in chaotic systems with machine learning

Model-Free Prediction of Chaotic Systems Using High Efficient Next-generation Reservoir Computing

Predicting Chaotic System Behavior using Machine Learning Techniques

Synchronization of Chaotic Systems and Their Machine-Learning Models

Reservoir Computing with Diverse Timescales for Prediction of Multiscale Dynamics

Learning spatiotemporal chaos using next-generation reservoir computing

Modeling Chaotic Systems: Dynamical Equations Vs Machine Learning Approach

An asynchronously deep reservoir computing for predicting chaotic time series

Application of next-generation reservoir computing for predicting chaotic systems from partial observations

Generalized Extended Kalman Filter For Prediction Of Chaotic Time-Series With Intermittent Failures

Improved Long-Term Prediction of Chaos Using Reservoir Computing Based on Stochastic Spin-Orbit Torque Devices

Oscillations enhance time-series prediction in reservoir computing with feedback

Temporal Convolution Derived Multi-Layered Reservoir Computing

Optimizing Echo State Networks for Enhancing Large Prediction Horizons of Chaotic Time Series

A Hybrid Proper Orthogonal Decomposition and Next Generation Reservoir Computing Approach for High-Dimensional Chaotic Prediction: Application to Flow-Induced Vibration of Tube Bundles

Prediction of chaotic dynamics and extreme events: A recurrence-free quantum reservoir computing approach

Prediction of Unobserved Bifurcation by Unsupervised Extraction of Slowly Time-Varying System Parameter Dynamics from Time Series Using Reservoir Computing

Stabilizing machine learning prediction of dynamics: Novel noise-inspired regularization tested with reservoir computing