Abstract:Time series data, characterized by its intrinsic long and short-range dependencies, poses a unique challenge across analytical applications. While Transformer-based models excel at capturing long-range dependencies, they face limitations in noise sensitivity, computational efficiency, and overfitting with smaller datasets. In response, we introduce a novel Time Series Lightweight Adaptive Network (TSLANet), as a universal convolutional model for diverse time series tasks. Specifically, we propose an Adaptive Spectral Block, harnessing Fourier analysis to enhance feature representation and to capture both long-term and short-term interactions while mitigating noise via adaptive thresholding. Additionally, we introduce an Interactive Convolution Block and leverage self-supervised learning to refine the capacity of TSLANet for decoding complex temporal patterns and improve its robustness on different datasets. Our comprehensive experiments demonstrate that TSLANet outperforms state-of-the-art models in various tasks spanning classification, forecasting, and anomaly detection, showcasing its resilience and adaptability across a spectrum of noise levels and data sizes. The code is available at

What problem does this paper attempt to address?

This paper attempts to address the unique challenges of time series data in analytical applications, particularly in capturing long and short-term dependencies, noise sensitivity, computational efficiency, and overfitting on small datasets. Specifically, although Transformer-based models excel in capturing long-term dependencies, they have limitations in noise sensitivity, computational efficiency, and overfitting on small datasets. To tackle these issues, the authors propose a new model called TSLANet (TimeSeries Lightweight Adaptive Network). ### Main Issues 1. **Noise Sensitivity**: Time series data often contains a lot of noise, which can affect the model's performance. 2. **Computational Efficiency**: Transformer models have high computational costs when processing large-scale time series data. 3. **Overfitting**: Transformer models are prone to overfitting on small datasets. 4. **Long and Short-term Dependencies**: Existing models are inadequate in capturing the long and short-term dependencies in time series data. ### Solutions 1. **Adaptive Spectral Block (ASB)**: - Enhances feature representation using Fourier transform while capturing long and short-term interactions. - Reduces noise through an adaptive thresholding method. 2. **Interactive Convolution Block (ICB)**: - Uses convolution kernels of different sizes to capture complex temporal patterns. - Improves the model's decoding ability on different datasets through self-supervised learning. 3. **Lightweight Architecture**: - Adopts a lightweight design for higher computational efficiency and speed. - Utilizes Fast Fourier Transform (FFT) operations to achieve O(NlogN) complexity. ### Experimental Results - **Classification Tasks**: TSLANet outperforms existing Transformer and MLP models in multiple time series classification tasks. - **Prediction Tasks**: In multivariate prediction tasks, TSLANet achieves good results across various prediction lengths. - **Anomaly Detection Tasks**: TSLANet also performs well in anomaly detection tasks, especially when dealing with complex and noisy data. ### Summary TSLANet effectively addresses the issues of long and short-term dependencies, noise sensitivity, and computational efficiency in time series data by combining convolution operations and frequency domain analysis. It is suitable for various time series tasks, including classification, prediction, and anomaly detection.

TSLANet: Rethinking Transformers for Time Series Representation Learning

TFEformer: Temporal Feature Enhanced Transformer for Multivariate Time Series Forecasting

TLNets: Transformation Learning Networks for long-range time-series prediction

Time-Transformer: Integrating Local and Global Features for Better Time Series Generation

A Time Series is Worth 64 Words: Long-term Forecasting with Transformers

Itransformer: Inverted Transformers Are Effective for Time Series Forecasting

AutoTransformer: Automatic Transformer Architecture Design for Time Series Classification

Enhanced Linear and Vision Transformer-Based Architectures for Time Series Forecasting

A Temporal Linear Network for Time Series Forecasting

Leveraging 2D Information for Long-term Time Series Forecasting with Vanilla Transformers

Are Transformers Effective for Time Series Forecasting?

sTransformer: A Modular Approach for Extracting Inter-Sequential and Temporal Information for Time-Series Forecasting

Transformers in Time Series: A Survey

Time Series Representation Models

AutoLDT: a lightweight spatio-temporal decoupling transformer framework with AutoML method for time series classification

ModernTCN: A Modern Pure Convolution Structure for General Time Series Analysis.

ConvLSTMTransNet: A Hybrid Deep Learning Approach for Internet Traffic Telemetry

UnetTSF: A Better Performance Linear Complexity Time Series Prediction Model

Learning Abstract Snippet Detectors with Temporal Embedding in Convolutional Neural Networks

VisionTwinNet: Gated Clarity Enhancement Paired With Light-Robust CD Transformers