Abstract:Reservoir computing is a computational paradigm derived from recurrent neural network models. One of its most representative technique is the Echo State Network (ESN), which is usually composed by two salient components: reservoir and readout. The former is responsible by mapping temporal (or sequential) inputs into a high-dimensional space and the latter aims at learning the patterns in such a new space. Despite ESN has attracted a lot of attention nowadays, most works in the literature are focused on the development of additional features to the model, while there is a considerable lack of investigations related to understand and evaluate some of its inherent concepts as well as their relationships. In this paper we address such a limitation by investigating ESN components related to the reservoir structure and readout layer. To be specific, we evaluate regular and small-world network models besides the extensively adopted random one, and also analyze a total of eight classification techniques instead of considering just the few techniques largely adopted in the literature (mostly linear ones). In order to consistently evaluate the alternative ESN methods, we analyzed a wide range of parameters in both reservoir and readout layers through of several experiments with five real-world data sets. The results revealed that some problems can be considerably benefited from some level of organization in the reservoir, such as those provided by regular or small-world network models; and that the non-linear support vector machine classifier achieved the best predictive performance, although it was statistically comparable with the k-nearest neighbors one, which has much smaller time complexity. Interestingly, such findings may make the adoption of ESN methods more efficient from the point of view of embedded systems and large scale problems.

Imposing Connectome-Derived Topology on an Echo State Network

Determining Reservoir Topologies from Short-Term Memory in Echo State Networks.

Accessing the topological properties of human brain functional sub-circuits in Echo State Networks

The Architecture of Dynamic Reservoir in the Echo State Network

Regular Echo State Networks

The Combination of Circle Topology and Leaky Integrator Neurons Remarkably Improves the Performance of Echo State Network on Time Series Prediction

Performance Evaluation of New Echo State Networks Based on Complex Network

The exponential distance rule based network model predicts topology and reveals functionally relevant properties of the Drosophila projectome

Exploiting Multiple Timescales in Hierarchical Echo State Networks

Modelling and evaluating restricted ESNs on single- and multi-timescale problems

Echo state networks for modeling turbulent convection

Direct model of memory properties and the linear reservoir topologies in echo state networks

A Double-Cycle Echo State Network Topology for Time Series Prediction.

Feature-Based Echo-State Networks: A Step Towards Interpretability and Minimalism in Reservoir Computer

Echo state network based on improved fruit fly optimization algorithm for chaotic time series prediction

Multifunctionality in a Connectome-Based Reservoir Computer

Embedding and approximation theorems for echo state networks

Echo State network for coarsening dynamics of charge density waves

Improving the Performance of Echo State Networks Through Feedback

Effects of connectivity structure of complex echo state network on its prediction performance for nonlinear time series

Effect of hybrid circle reservoir injected with wavelet-neurons on performance of echo state network