Abstract:Reservoir computing is a computational paradigm derived from recurrent neural network models. One of its most representative technique is the Echo State Network (ESN), which is usually composed by two salient components: reservoir and readout. The former is responsible by mapping temporal (or sequential) inputs into a high-dimensional space and the latter aims at learning the patterns in such a new space. Despite ESN has attracted a lot of attention nowadays, most works in the literature are focused on the development of additional features to the model, while there is a considerable lack of investigations related to understand and evaluate some of its inherent concepts as well as their relationships. In this paper we address such a limitation by investigating ESN components related to the reservoir structure and readout layer. To be specific, we evaluate regular and small-world network models besides the extensively adopted random one, and also analyze a total of eight classification techniques instead of considering just the few techniques largely adopted in the literature (mostly linear ones). In order to consistently evaluate the alternative ESN methods, we analyzed a wide range of parameters in both reservoir and readout layers through of several experiments with five real-world data sets. The results revealed that some problems can be considerably benefited from some level of organization in the reservoir, such as those provided by regular or small-world network models; and that the non-linear support vector machine classifier achieved the best predictive performance, although it was statistically comparable with the k-nearest neighbors one, which has much smaller time complexity. Interestingly, such findings may make the adoption of ESN methods more efficient from the point of view of embedded systems and large scale problems.

Recursive Least Squares Algorithm with Adaptive Forgetting Factor Based on Echo State Network

Improved Recursive Least Squares Algorithm Based on Echo State Neural Network for Nonlinear System Identification

Long-Short Term Echo State Network for Time Series Prediction

Self-Adaptive Particle Swarm Optimization-Based Echo State Network for Time Series Prediction

Memory augmented echo state network for time series prediction

Fuzzy Echo State Networks

An on-line training algorithm for echo state networks with extended kalman filter

Echo State Networks for Mobile Robot Modeling and Control

Ensemble and Pre-Training Approach for Echo State Network and Extreme Learning Machine Models

Attention Based Echo State Network

Variable Forgetting Factor Mechanisms for Diffusion Recursive Least Squares Algorithm in Sensor Networks.

A Gated Recurrent Unit Based Echo State Network

Recursive Least Squares-Algorithm-Based Normalized Adaptive Minimum Symbol Error Rate Equalizer.

A Review of Designs and Applications of Echo State Networks

A Modified Echo State Network Based Remaining Useful Life Estimation Approach

A Systematic Review of Echo State Networks from Design to Application

Low-complexity Correlated Time-Averaged Variable Forgetting Factor Mechanism for Diffusion RLS Algorithm in Sensor Networks

Echo State Networks with Self-Normalizing Activations on the Hyper-Sphere

Design and Application of Adaptive Sparse Deep Echo State Network

Review on Echo State Networks

Regular Echo State Networks