Abstract:Echo state networks (ESNs) are a new approach to recurrent neural networks (RNNs) that have been successfully applied in many domains. Nevertheless, an ESN is a predictive model rather than a classifier, and methods to employ ESNs in time series classification (TSC) tasks have not yet been fully explored. In this paper, we propose a novel ESN approach named functional echo state network (FESN) for time series classification. The basic idea behind FESN is to replace the numeric variable output weights of an ESN with time-varying output-weight functions and introduce a temporal aggregation operator to the output layer that can project temporal signals into discrete class labels, thereby transforming the ESN from a predictive model into a true classifier. Subsequently, to learn the output-weight functions, a spatio-temporal aggregation learning algorithm is proposed based on orthogonal function basis expansion. By leveraging the nonlinear mapping capacity of a reservoir and the accumulation of temporal information in the time domain, FESN can not only enhance the separability of different classes in a high-dimensional functional space but can also consider the relative importance of temporal data at different time steps according to dynamic output-weight functions. Theoretical analyses and experiments on an extensive set of UCR data were conducted on FESN. The results show that FESN yields better performance than single-algorithm methods, has comparable accuracy with ensemble-based methods and exhibits acceptable computational complexity. Interestingly, for some time series datasets, we visualized some interpretable features extracted by FESN via specific patterns within the output-weight functions.

Subspace Echo State Network for Multivariate Time Series Prediction

Predicting Multivariate Time Series Using Subspace Echo State Network

Hierarchical Echo State Network with Sparse Learning: A Method for Multidimensional Chaotic Time Series Prediction.

Long-Short Term Echo State Network for Time Series Prediction

Design of Sparse Bayesian Echo State Network for Time Series Prediction

T-distributed stochastic neighbor embedding echo state network with state matrix dimensionality reduction for time series prediction

Sequence Prediction with Different Dimensions Based on Two Novel Deep Echo State Network Models

Functional Echo State Network for Time Series Classification.

Time Series Prediction Using Time-Series Decomposition and Multi-reservoirs Echo State Network

Multivariate Nonlinear Time Series Prediction by Chain-Structure Echo State Network

Echo State Networks with Double-Reservoir for Time-Series Prediction

Hybrid Regularized Echo State Network for Multivariate Chaotic Time Series Prediction

Delayed Deep Echo State Network and Its Application on Time Series Prediction

Broad Echo State Network for Multivariate Time Series Prediction.

A Double-Cycle Echo State Network Topology for Time Series Prediction.

Multi-state Delayed Echo State Network with Empirical Wavelet Transform for Time Series Prediction

Error-Driven Chained Multiple-Subnetwork Echo State Network for Time-Series Prediction

Self-Adaptive Particle Swarm Optimization-Based Echo State Network for Time Series Prediction

Dynamical Regularized Echo State Network for Time Series Prediction

Adaptive Lasso Echo State Network for Time Series Prediction

Chaotic Time Series Prediction Using a Novel Echo State Network Model with Input Reconstruction, Bayesian Ridge Regression and Independent Component Analysis