Abstract:Time series classification finds widespread applications in civil, industrial, and military fields, while the classification performance of time series models has been improving with the recent development of deep learning. However, the issues of feature extraction effectiveness, model complexity, and model design uncertainty constrain the further development of time series classification. To address the above issues, we propose a Lightweight Spatio-Temporal Decoupling Transformer framework based on Automated Machine Learning technique (AutoLDT). The framework proposes a novel lightweight Transformer with fuzzy position encoding, TS-separable linear self-attention mechanism, and convolutional feedforward network, which mine the temporal and spatial features, as well as the local and global relationship of time series. Fuzzy positional encoding integrates fuzzy ideas to enhance the generalization performance of model information mining. TS-separable linear self-attention mechanism and convolutional feedforward network achieve feature extraction in a lightweight way by decoupling temporal and spatial features of time series. Notably, we adopt the Covariance Matrix Adaptation Evolution Strategy and global adaptive pruning technique to realize automated network structure design, which further improves the model training efficiency and automation, and avoids the uncertainty problem of network design. Finally, we validate the effectiveness of the proposed framework on publicly available UCR and UEA time series datasets. The experimental results show that the proposed framework not only improves the model classification performance in a lightweight way but also dramatically improves the model training efficiency.

AutoTransformer: Automatic Transformer Architecture Design for Time Series Classification

AutoLDT: a lightweight spatio-temporal decoupling transformer framework with AutoML method for time series classification

TransNAS-TSAD: harnessing transformers for multi-objective neural architecture search in time series anomaly detection

MSTAR: Multi-Scale Backbone Architecture Search for Timeseries Classification

TSLANet: Rethinking Transformers for Time Series Representation Learning

Decompose Auto-Transformer Time Series Anomaly Detection for Network Management

NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecasting

Time-Transformer: Integrating Local and Global Features for Better Time Series Generation

Itransformer: Inverted Transformers Are Effective for Time Series Forecasting

AutoTrans: Automating Transformer Design via Reinforced Architecture Search

RNTS: Robust Neural Temporal Search for Time Series Classification

FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series Classification

InceptionTime: Finding AlexNet for time series classification

sTransformer: A Modular Approach for Extracting Inter-Sequential and Temporal Information for Time-Series Forecasting

Transformers in Time-series Analysis: A Tutorial

Two Steps Forward and One Behind: Rethinking Time Series Forecasting with Deep Learning

Multivariate Time Series Classification with Crucial Timestamps Guidance

CTCTime: A New Model for Unidimensional Time Series Classification

Transformers in Time Series: A Survey

TCF-Trans: Temporal Context Fusion Transformer for Anomaly Detection in Time Series