Abstract:We introduce and detail an atypical neural network architecture, called time elastic neural network (teNN), for multivariate time series classification. The novelty compared to classical neural network architecture is that it explicitly incorporates time warping ability, as well as a new way of considering attention. In addition, this architecture is capable of learning a dropout strategy, thus optimizing its own architecture.Behind the design of this architecture, our overall objective is threefold: firstly, we are aiming at improving the accuracy of instance based classification approaches that shows quite good performances as far as enough training data is available. Secondly we seek to reduce the computational complexity inherent to these methods to improve their scalability. Ideally, we seek to find an acceptable balance between these first two criteria. And finally, we seek to enhance the explainability of the decision provided by this kind of neural architecture.The experiment demonstrates that the stochastic gradient descent implemented to train a teNN is quite effective. To the extent that the selection of some critical meta-parameters is correct, convergence is generally smooth and fast.While maintaining good accuracy, we get a drastic gain in scalability by first reducing the required number of reference time series, i.e. the number of teNN cells required. Secondly, we demonstrate that, during the training process, the teNN succeeds in reducing the number of neurons required within each cell. Finally, we show that the analysis of the activation and attention matrices as well as the reference time series after training provides relevant information to interpret and explain the classification results.The comparative study that we have carried out and which concerns around thirty diverse and multivariate datasets shows that the teNN obtains results comparable to those of the state of the art, in particular similar to those of a network mixing LSTM and CNN architectures for example.

Towards a universal neural network encoder for time series

Improving Time Series Encoding with Noise-Aware Self-Supervised Learning and an Efficient Encoder

Time Elastic Neural Networks

TimeNet: Pre-trained deep recurrent neural network for time series classification

Deep Temporal Convolution Network for Time Series Classification

Encoding temporal information in deep convolution neural network

A Brain-Inspired Spiking Neural Network Model with Temporal Encoding and Learning

TSLANet: Rethinking Transformers for Time Series Representation Learning

Prediction Under Uncertainty with Error-Encoding Networks

A Temporal Encoding Method Based on Expansion Representation

InceptionTime: Finding AlexNet for time series classification

Learning Abstract Snippet Detectors with Temporal Embedding in Convolutional Neural Networks

Combining contextual neural networks for time series classification

Time Scale Network: A Shallow Neural Network For Time Series Data

A Novel Encoder-Decoder Model for Multivariate Time Series Forecasting

LETS-C: Leveraging Language Embedding for Time Series Classification

Time-series modeling with undecimated fully convolutional neural networks

Time Series Compression Based on Adaptive Piecewise Recurrent Autoencoder

Get a New Perspective on EEG: Convolutional Neural Network Encoders for Parametric t-SNE

Multi-Scale Convolutional Neural Networks for Time Series Classification