Abstract:The echo state network (ESN) is a recurrent neural network that has yielded state-of-the-art results in many areas owing to its rapid learning ability and the fact that the weights of input neurons and hidden neurons are fixed throughout the learning process. However, the setting procedure for initializing the ESN's recurrent structure may lead to difficulties in designing a sound reservoir that matches a specific task. This paper proposes an improved pre-training method to adjust the model's parameters and topology to obtain an adaptive reservoir for a given application. Two strategies, namely global random selection and ensemble training, are introduced to pre-train the randomly initialized ESN model. Specifically, particle swarm optimization is applied to optimize chosen fixed and global weight values within the network, and the reliability and stability of the pre-trained model are enhanced by employing the ensemble training strategy. In addition, we test the feasibility of the model for time series prediction on six benchmarks and two real-life datasets. The experimental results show a clear enhancement in the ESN learning results. Furthermore, the proposed global random selection and ensemble training strategies are also applied to pre-train the extreme learning machine (ELM), which has a similar training process to the ESN model. Numerical experiments are subsequently carried out on the above-mentioned eight datasets. The experimental findings consistently show that the performance of the proposed pre-trained ELM model is also improved significantly. The suggested two strategies can thus enhance the ESN and ELM models' prediction accuracy and adaptability.

Online Sequential Echo State Network with Sparse RLS Algorithm for Time Series Prediction.

Long-Short Term Echo State Network for Time Series Prediction

Self-Adaptive Particle Swarm Optimization-Based Echo State Network for Time Series Prediction

Recursive Least Squares Algorithm with Adaptive Forgetting Factor Based on Echo State Network

Hybrid Regularized Echo State Network for Multivariate Chaotic Time Series Prediction

Echo State Network Based on Phase Space Reconstruction: El Niño 3 Index Forecasting

Improved Recursive Least Squares Algorithm Based on Echo State Neural Network for Nonlinear System Identification

Grey Wolf Optimization–Based Deep Echo State Network for Time Series Prediction

Design and Application of Adaptive Sparse Deep Echo State Network

T-distributed stochastic neighbor embedding echo state network with state matrix dimensionality reduction for time series prediction

Distributed Sparse Recursive Least-Squares over Networks

A novel time series prediction method based on pooling compressed sensing echo state network and its application in stock market

Fractional-integer-order echo state network for time series prediction

Online robust echo state broad learning system

Nonstationary Time Series Prediction Based on Deep Echo State Network Tuned by Bayesian Optimization

Ensemble and Pre-Training Approach for Echo State Network and Extreme Learning Machine Models

Memory augmented echo state network for time series prediction

Time Series Prediction of ESN Based on Chebyshev Mapping and Strongly Connected Topology

Semi-supervised Echo State Network with Partial Correlation Pruning for Time-Series Variables Prediction in Industrial Processes

Chain-Structure Echo State Network With Stochastic Optimization: Methodology and Application

Chaotic Time Series Prediction Using a Novel Echo State Network Model with Input Reconstruction, Bayesian Ridge Regression and Independent Component Analysis