Abstract:Irregularly sampled multivariate time series (ISMTS) are prevalent in reality. Due to their non-uniform intervals between successive observations and varying sampling rates among series, the channel-independent (CI) strategy, which has been demonstrated more desirable for complete multivariate time series forecasting in recent studies, has failed. This failure can be further attributed to the sampling sparsity, which provides insufficient information for effective CI learning, thereby reducing its capacity. When we resort to the channel-dependent (CD) strategy, even higher capacity cannot mitigate the potential loss of diversity in learning similar embedding patterns across different channels. We find that existing work considers CI and CD strategies to be mutually exclusive, primarily because they apply these strategies to the global channel. However, we hold the view that channel strategies do not necessarily have to be used globally. Instead, by appropriately applying them locally and globally, we can create an opportunity to take full advantage of both strategies. This leads us to introduce the Channel Harmony ISMTS Transformer (TimeCHEAT), which utilizes the CD locally and the CI globally. Specifically, we segment the ISMTS into sub-series level patches. Locally, the CD strategy aggregates information within each patch for time embedding learning, maximizing the use of relevant observations while reducing long-range irrelevant interference. Here, we enhance generality by transforming embedding learning into an edge weight prediction task using bipartite graphs, eliminating the need for special prior knowledge. Globally, the CI strategy is applied across patches, allowing the Transformer to learn individualized attention patterns for each channel. Experimental results indicate our proposed TimeCHEAT demonstrates competitive SOTA performance across three mainstream tasks.

CaSS: A Channel-aware Self-supervised Representation Learning Framework for Multivariate Time Series Classification

Multi-view Self-Supervised Contrastive Learning for Multivariate Time Series

DCS: Debiased Contrastive Learning with Weak Supervision for Time Series Classification

A Deep Multi-Task Representation Learning Method for Time Series Classification and Retrieval.

MICOS: Mixed supervised contrastive learning for multivariate time series classification

Self-supervised Contrastive Representation Learning for Semi-supervised Time-Series Classification

TS-TFSIAM: Time-series Self-Supervised Learning with Time-Frequency SiameseNet

Multi-view self-supervised learning for multivariate variable-channel time series

Unsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral Fusion.

TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignment

Attention Based CNN-LSTM Network for Anomaly Pattern Classification of Multivariate Time Series

Multi-scale Self-Supervised Representation Learning with Temporal Alignment for Multi-Rate Time Series Modeling

Multi-Task Self-Supervised Time-Series Representation Learning

FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series Classification

Self-supervised Representation Learning for Time Series Via Temporal Contrasting and Transformation

Multiscale spatial‐temporal transformer with consistency representation learning for multivariate time series classification

A Multi-View Multi-Task Learning Framework for Multi-Variate Time Series Forecasting

VSFormer: Value and Shape-Aware Transformer with Prior-Enhanced Self-Attention for Multivariate Time Series Classification

TimeCHEAT: A Channel Harmony Strategy for Irregularly Sampled Multivariate Time Series Analysis