Abstract:Conventional time series classification approaches based on bags of patterns or shapelets face significant challenges in dealing with a vast amount of feature candidates from high-dimensional multivariate data. In contrast, deep neural networks can learn low-dimensional features efficiently, and in particular, Convolutional Neural Networks (CNN) have shown promising results in classifying Multivariate Time Series (MTS) data. A key factor in the success of deep neural networks is this astonishing expressive power. However, this power comes at the cost of complex, black-boxed models, conflicting with the goals of building reliable and human-understandable models. An essential criterion in understanding such predictive deep models involves quantifying the contribution of time-varying input variables to the classification. Hence, in this work, we introduce a new framework for interpreting multivariate time series data by extracting and clustering the input representative patterns that highly activate CNN neurons. This way, we identify each signal's role and dependencies, considering all possible combinations of signals in the MTS input. Then, we construct a graph that captures the temporal relationship between the extracted patterns for each layer. An effective graph merging strategy finds the connection of each node to the previous layer's nodes. Finally, a graph embedding algorithm generates new representations of the created interpretable time-series features. To evaluate the performance of our proposed framework, we run extensive experiments on eight datasets of the UCR/UEA archive, along with HAR and PAM datasets. The experiments indicate the benefit of our time-aware graph-based representation in MTS classification while enriching them with more interpretability.

Encoding Time Series as Images for Visual Inspection and Classification Using Tiled Convolutional Neural Networks

Imaging Time-Series to Improve Classification and Imputation

Spatially Encoding Temporal Correlations to Classify Temporal Data Using Convolutional Neural Networks

Sensor Classification Using Convolutional Neural Network by Encoding Multivariate Time Series as Two-Dimensional Colored Images

MTS2Graph: Interpretable Multivariate Time Series Classification with Temporal Evolving Graphs

Data-Folding and Hyperspace Coding for Multi-Dimensonal Time-Series Data Imaging

Lagged encoding for image‐based time series classification using convolutional neural networks

Timage -- A Robust Time Series Classification Pipeline

Encoding Time Series as Multi-Scale Signed Recurrence Plots for Classification Using Fully Convolutional Networks

A Deep Learning Framework for Time Series Classification Using Relative Position Matrix and Convolutional Neural Network.

Encoding Candlesticks as Images for Patterns Classification Using Convolutional Neural Networks

Emotion Recognition Based on Gramian Encoding Visualization

A T-CNN time series classification method based on Gram matrix

A Temporal Encoding Method Based on Expansion Representation

From Pixels to Predictions: Spectrogram and Vision Transformer for Better Time Series Forecasting

Multi-scale Harmonic Mean Time Surfaces for Event-based Object Classification

Encoding candlesticks as images for pattern classification using convolutional neural networks

Encoding Time-Series Ground Motions as Images for Convolutional Neural Networks-Based Seismic Damage Evaluation

XCM: an Explainable Convolutional Neural Network for Multivariate Time Series Classification.

TCGAN: Convolutional Generative Adversarial Network for Time Series Classification and Clustering.

Classification of Arrhythmia in Time Series ECG Signals Using Image Encoding And Convolutional Neural Networks