Abstract:Self-supervised learning has garnered significant attention for its ability to learn meaningful representations. Recent advancements have introduced self-supervised methods for time series forecasting. However, these efforts have faced limitations due to two primary drawbacks. Firstly, these approaches often borrow techniques from vision and language domains without adequately addressing the unique temporal dependencies inherent in time series data. Secondly, time series often show that the distribution shifts over time, which makes accurate forecasting challenging. In response to these issues, we propose TempSSL, a self-supervised learning framework designed for time series forecasting. TempSSL divides the time series data into context (history data) and target (future data), employing two pre-training strategies: (1) Temporal Masked Modeling (TMM) designed to capture temporal dependencies by reconstructing future time series based on historical context; (2) Temporal Contrastive Learning (TCL) employs context and target as positive samples to enhance discriminative representations and mitigate distribution shifts within the time series. TempSSL's innovation lies in two key aspects. Firstly, it underscores the importance of temporal dependencies for time series forecasting by designing specific pre-training tasks. Secondly, it effectively integrates contrastive learning and masked modeling, leveraging their respective strengths to develop time series representation with strong instance discriminability and local perceptibility. Extensive experiments across seven widely used benchmark datasets demonstrate that TempSSL consistently outperforms existing self-supervised and end-to-end forecasting methods, achieving improvements ranging from 1.92% ∼ 78.12%. Additionally, TempSSL's practical effectiveness is further demonstrated through successful application in natural gas demand forecasting.

Temporal Feature Selection on Networked Time Series

Learning Shapelet Patterns from Network-Based Time Series

New Word Identification in Social Network Text Based on Time Series Information

Time Series Feature Learning with Labeled and Unlabeled Data

RTFN: A robust temporal feature network for time series classification

Discriminative shapelet learning via temporal clustering and matrix factorization

A Shapelet Learning Method for Time Series Classification

A Deep Multi-Task Representation Learning Method for Time Series Classification and Retrieval.

Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks

RTFN: Robust Temporal Feature Network

A Feature Segment Based Time Series Classification Algorithm

TV-Net: Temporal-Variable feature harmonizing Network for multivariate time series classification and interpretation

Unsupervised Hierarchical Feature Selection on Networked Data.

Model Selection with a Shapelet-based Distance Measure for Multi-source Transfer Learning in Time Series Classification

Scalable Discovery of Time-Series Shapelets

LeRet: Language-Empowered Retentive Network for Time Series Forecasting

Rethinking self-supervised learning for time series forecasting: A temporal perspective

Temporal Network Representation Learning via Historical Neighborhoods Aggregation

Longitudinal LASSO: Jointly Learning Features and Temporal Contingency for Outcome Prediction

Learning Abstract Snippet Detectors with Temporal Embedding in Convolutional Neural Networks

Local Sparse Discriminative Feature Selection