Abstract:Echo state networks (ESNs) are a new approach to recurrent neural networks (RNNs) that have been successfully applied in many domains. Nevertheless, an ESN is a predictive model rather than a classifier, and methods to employ ESNs in time series classification (TSC) tasks have not yet been fully explored. In this paper, we propose a novel ESN approach named functional echo state network (FESN) for time series classification. The basic idea behind FESN is to replace the numeric variable output weights of an ESN with time-varying output-weight functions and introduce a temporal aggregation operator to the output layer that can project temporal signals into discrete class labels, thereby transforming the ESN from a predictive model into a true classifier. Subsequently, to learn the output-weight functions, a spatio-temporal aggregation learning algorithm is proposed based on orthogonal function basis expansion. By leveraging the nonlinear mapping capacity of a reservoir and the accumulation of temporal information in the time domain, FESN can not only enhance the separability of different classes in a high-dimensional functional space but can also consider the relative importance of temporal data at different time steps according to dynamic output-weight functions. Theoretical analyses and experiments on an extensive set of UCR data were conducted on FESN. The results show that FESN yields better performance than single-algorithm methods, has comparable accuracy with ensemble-based methods and exhibits acceptable computational complexity. Interestingly, for some time series datasets, we visualized some interpretable features extracted by FESN via specific patterns within the output-weight functions.

Hierarchical Echo State Network with Sparse Learning: A Method for Multidimensional Chaotic Time Series Prediction.

Multistep-Ahead Chaotic Time Series Prediction Based on Hierarchical Echo State Network With Augmented Random Features

Hierarchical Delay-Memory Echo State Network: A Model Designed for Multi-Step Chaotic Time Series Prediction.

Hybrid Regularized Echo State Network for Multivariate Chaotic Time Series Prediction

Long-Short Term Echo State Network for Time Series Prediction

Physics-informed hierarchical echo state network for predicting the dynamics of chaotic systems

Subspace Echo State Network for Multivariate Time Series Prediction

Design of Sparse Bayesian Echo State Network for Time Series Prediction

Adaptive Levenberg-Marquardt Algorithm Based Echo State Network for Chaotic Time Series Prediction

L1/2 Norm Regularized Echo State Network for Chaotic Time Series Prediction.

Predicting Multivariate Time Series Using Subspace Echo State Network

LM Algorithm in Echo State Network for Chaotic Time Series Prediction

Chaotic Time Series Prediction Using a Novel Echo State Network Model with Input Reconstruction, Bayesian Ridge Regression and Independent Component Analysis

Multi-steps prediction of chaotic time series based on echo state network

T-distributed stochastic neighbor embedding echo state network with state matrix dimensionality reduction for time series prediction

Functional Echo State Network for Time Series Classification.

Adaptive Lasso Echo State Network for Time Series Prediction

Online Sequential Echo State Network with Sparse RLS Algorithm for Time Series Prediction.

Sequence Prediction with Different Dimensions Based on Two Novel Deep Echo State Network Models

Reservoir neural state reconstruction and chaotic time series prediction

Dynamical Regularized Echo State Network for Time Series Prediction