Abstract:Irregularly sampled multivariate time series (ISMTS) are prevalent in reality. Most existing methods treat ISMTS as synchronized regularly sampled time series with missing values, neglecting that the irregularities are primarily attributed to variations in sampling rates. In this paper, we introduce a novel perspective that irregularity is essentially relative in some senses. With sampling rates artificially determined from low to high, an irregularly sampled time series can be transformed into a hierarchical set of relatively regular time series from coarse to fine. We observe that additional coarse-grained relatively regular series not only mitigate the irregularly sampled challenges to some extent but also incorporate broad-view temporal information, thereby serving as a valuable asset for representation learning. Therefore, following the philosophy of learning that Seeing the big picture first, then delving into the details, we present the Multi-Scale and Multi-Correlation Attention Network (MuSiCNet) combining multiple scales to iteratively refine the ISMTS representation. Specifically, within each scale, we explore time attention and frequency correlation matrices to aggregate intra- and inter-series information, naturally enhancing the representation quality with richer and more intrinsic details. While across adjacent scales, we employ a representation rectification method containing contrastive learning and reconstruction results adjustment to further improve representation consistency. MuSiCNet is an ISMTS analysis framework that competitive with SOTA in three mainstream tasks consistently, including classification, interpolation, and forecasting.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve **the challenges in the analysis of irregularly - sampled multivariate time series (ISMTS)**. Specifically, the authors point out that existing methods usually regard ISMTS as synchronous regularly - sampled time series with missing values, ignoring that the irregularity is mainly caused by the change of sampling rates. This treatment may distort the internal relationships of the data and introduce noise, thus affecting the accuracy of analysis tasks. To solve these problems, the paper proposes a new framework named **MuSiCNet (Multi - Scale and Multi - Correlation Attention Network)**. This framework improves the representation learning of ISMTS in the following ways: 1. **Multi - scale learning**: The authors believe that irregularity is relative to some extent, and by artificially setting sampling rates from low to high, the irregularly - sampled time series can be transformed into a set of relatively regular time series from coarse to fine. This method not only alleviates the challenges of irregular sampling, but also incorporates more extensive temporal information, which is helpful for representation learning. 2. **Multi - correlation attention mechanism**: At each scale, MuSiCNet uses time - attention and frequency - correlation matrices to aggregate the information within and between sequences, thereby enhancing the representation quality. 3. **Cross - scale correction**: Between adjacent scales, MuSiCNet adopts the methods of contrastive learning and reconstruction result adjustment to further improve the consistency of the representation. 4. **Task generality**: MuSiCNet is not limited to specific analysis tasks, but attempts to propose a general model suitable for ISMTS analysis, including classification, interpolation and prediction. ### Formula summary - **Time - attention mechanism**: \[ A^T=\text{softmax}\left(\frac{Q^T K}{\sqrt{d_r}}\right) \] where \(Q\) and \(K\) are the query and key respectively, and \(d_r\) is the dimension. - **Reconstruction loss**: \[ \ell^{(l)}_{\text{recon}}=\sum_i \| M^{(l)} \odot (\hat{X}^{(l)}_{\text{reco}} - X^{(l)}) \|_2^2 \] where \(M^{(l)}\) is the mask of the \(l\)-th layer, and \(\odot\) represents the Hadamard product. - **Cross - scale alignment loss**: \[ \ell^{(l)}_{\text{adj}}=\sum_i \left\| \text{AvgPooling}_l(\hat{X}^{(l)}_{\text{reco}})_i - (\hat{X}^{(l - 1)}_{\text{reco}})_i \right\|_2^2 \] - **Contrastive learning loss**: \[ \ell^{(l)}_{\text{cons}}=-\sum_i \log \frac{\exp(h^{(l)}_i \cdot h^{(l - 1)}_i)}{\sum_{j = 1}^B \left[ \exp(h^{(l)}_i \cdot h^{(l - 1)}_j)+I[i\neq j] \exp(h^{(l)}_i \cdot h^{(l)}_j) \right]} \] where \(I\) is the indicator function and \(B\) is the batch size. ### Conclusion Through these innovations, MuSiCNet can perform excellently in multiple mainstream tasks (such as classification, interpolation and prediction), and has low time and space complexity, especially when dealing with unbalanced data sets. This makes MuSiCNet a reliable and efficient tool for ISMTS analysis.

MuSiCNet: A Gradual Coarse-to-Fine Framework for Irregularly Sampled Multivariate Time Series Analysis

Attention Based CNN-LSTM Network for Anomaly Pattern Classification of Multivariate Time Series

Improving Irregularly Sampled Time Series Learning with Time-Aware Dual-Attention Memory-Augmented Networks.

MSGNet: Learning Multi-Scale Inter-Series Correlations for Multivariate Time Series Forecasting

Order-Preserving Metric Learning for Mining Multivariate Time Series

CANet: Convolutional Attention-based Network for Multivariate Time Series Anomaly Detection

ReCTSi: Resource-efficient Correlated Time Series Imputation Via Decoupled Pattern Learning and Completeness-aware Attentions

DAMA-Net: A Novel Predictive Model for Irregularly Asynchronously and Sparsely Sampled Multivariate Time Series

TV-Net: Temporal-Variable feature harmonizing Network for multivariate time series classification and interpretation

Crossmodal Sequential Interaction Network for Hyperspectral and LiDAR Data Joint Classification

Revisiting Attention for Multivariate Time Series Forecasting

MICOS: Mixed supervised contrastive learning for multivariate time series classification

TapNet: Multivariate Time Series Classification with Attentional Prototypical Network

Multiscale Alternately Updated Clique Network for Hyperspectral Image Classification

A Multiscale Interactive Recurrent Network for Time-Series Forecasting

MCSDNet: Mesoscale Convective System Detection Network via Multi-scale Spatiotemporal Information

Recognize enhanced temporal-spatial-spectral features with a parallel multi-branch CNN and GRU

SMD-Net: Siamese Multi-Scale Difference-Enhancement Network for Change Detection in Remote Sensing

A SSM is Polymerized from Multivariate Time Series

MiSSNet: Memory-Inspired Semantic Segmentation Augmentation Network for Class-Incremental Learning in Remote Sensing Images

Learning Structured Components: Towards Modular and Interpretable Multivariate Time Series Forecasting