Abstract:Self-supervised learning has garnered significant attention for its ability to learn meaningful representations. Recent advancements have introduced self-supervised methods for time series forecasting. However, these efforts have faced limitations due to two primary drawbacks. Firstly, these approaches often borrow techniques from vision and language domains without adequately addressing the unique temporal dependencies inherent in time series data. Secondly, time series often show that the distribution shifts over time, which makes accurate forecasting challenging. In response to these issues, we propose TempSSL, a self-supervised learning framework designed for time series forecasting. TempSSL divides the time series data into context (history data) and target (future data), employing two pre-training strategies: (1) Temporal Masked Modeling (TMM) designed to capture temporal dependencies by reconstructing future time series based on historical context; (2) Temporal Contrastive Learning (TCL) employs context and target as positive samples to enhance discriminative representations and mitigate distribution shifts within the time series. TempSSL's innovation lies in two key aspects. Firstly, it underscores the importance of temporal dependencies for time series forecasting by designing specific pre-training tasks. Secondly, it effectively integrates contrastive learning and masked modeling, leveraging their respective strengths to develop time series representation with strong instance discriminability and local perceptibility. Extensive experiments across seven widely used benchmark datasets demonstrate that TempSSL consistently outperforms existing self-supervised and end-to-end forecasting methods, achieving improvements ranging from 1.92% ∼ 78.12%. Additionally, TempSSL's practical effectiveness is further demonstrated through successful application in natural gas demand forecasting.

An Active Learning-Based Incremental Deep-Broad Learning Algorithm for Unbalanced Time Series Prediction

TKS-BLS: Temporal Kernel Stationary Broad Learning System for Enhanced Modeling, Anomaly Detection, and Incremental Learning with Application to Ironmaking Processes

Effective LSTMs with Seasonal-Trend Decomposition and Adaptive Learning and Niching-Based Backtracking Search Algorithm for Time Series Forecasting

Deep Learning for Knowledge-Driven Ontology Stream Prediction.

Recurrent Broad Learning Systems for Time Series Prediction

An Online Incremental Learning Algorithm for Time Series

Sparse Deep Learning for Time Series Data: Theory and Applications

Adaptive Memory Broad Learning System for Unsupervised Time Series Anomaly Detection

Deep Time Series Forecasting Models: A Comprehensive Survey

DAFA-BiLSTM: Deep Autoregression Feature Augmented Bidirectional LSTM network for time series prediction

Deep Learning with Long Short-Term Memory for Time Series Prediction

Time Series Supplier Allocation via Deep Black-Litterman Model

A Survey of Deep Learning and Foundation Models for Time Series Forecasting

A Review of Deep Learning Models for Time Series Prediction

An Active Learning Framework with a Class Balancing Strategy for Time Series Classification

Modeling Extreme Events in Time Series Prediction

Deep Learning for Time Series Forecasting: A Survey

Is Single Enough? A Joint Spatiotemporal Feature Learning Framework for Multivariate Time Series Prediction

Rethinking self-supervised learning for time series forecasting: A temporal perspective

Robust Incremental Broad Learning System for Data Streams of Uncertain Scale

Deep Incremental Learning of Imbalanced Data for Just-In-Time Software Defect Prediction