Abstract:Temporal distribution shifts are ubiquitous in time series data. One of the most popular methods assumes that the temporal distribution shift occurs uniformly to disentangle the stationary and nonstationary dependencies. But this assumption is difficult to meet, as we do not know when the distribution shifts occur. To solve this problem, we propose to learn IDentifiable latEnt stAtes (IDEA) to detect when the distribution shifts occur. Beyond that, we further disentangle the stationary and nonstationary latent states via sufficient observation assumption to learn how the latent states change. Specifically, we formalize the causal process with environment-irrelated stationary and environment-related nonstationary variables. Under mild conditions, we show that latent environments and stationary/nonstationary variables are identifiable. Based on these theories, we devise the IDEA model, which incorporates an autoregressive hidden Markov model to estimate latent environments and modular prior networks to identify latent states. The IDEA model outperforms several latest nonstationary forecasting methods on various benchmark datasets, highlighting its advantages in real-world scenarios.

What problem does this paper attempt to address?

The paper primarily addresses the issue of non-stationary time series forecasting and proposes a new method to overcome the limitations present in existing techniques. Specifically, the paper attempts to solve the following key problems: 1. **Handling Changes in Time Distribution**: Non-stationarity, or changes in time distribution, is common in time series data, posing a challenge to the generalization ability of predictive models over time. 2. **Limitations of Existing Methods**: - Assuming uniform distribution changes or instance-based partitioning, which is often too idealistic in practical applications. - Existing methods struggle to accurately separate stable and non-stable dependencies within the time series. 3. **Proposed New Method**: - **Detecting Time Points of Distribution Changes**: First, determine when distribution changes occur. - **Separating Stable and Non-Stable Latent States**: Further distinguish how these latent states change over time. The paper theoretically proves that the latent environment and stable/non-stable variables can be identified and designs a model named IDEA based on these theories. This model combines an autoregressive hidden Markov model to estimate the latent environment and uses a modular prior network architecture to identify stable and non-stable latent variables. ### Summary - **Target Problem**: Address the issue of non-stationary time series forecasting, particularly how to effectively handle non-stationarity in time series data. - **Solution**: Propose the IDEA model, which can detect when changes in time distribution occur and separate stable and non-stable latent states. - **Theoretical Contribution**: Prove the identifiability of the latent environment and stable/non-stable variables. - **Experimental Validation**: Validate the effectiveness and superiority of the IDEA model through experimental results on synthetic and real-world datasets. In summary, this paper aims to improve the performance of non-stationary time series forecasting tasks by proposing a new method, with a particular focus on effectively distinguishing and utilizing the stable and non-stable characteristics in time series data.

When and How: Learning Identifiable Latent States for Nonstationary Time Series Forecasting

Causal Discovery and Forecasting in Nonstationary Environments with State-Space Models

Learning Latent Causal Dynamics

Temporally Disentangled Representation Learning under Unknown Nonstationarity

Learning Interpretable Deep State Space Model for Probabilistic Time Series Forecasting

On the Identification of Temporally Causal Representation with Instantaneous Dependence

Addressing Distribution Shift in Time Series Forecasting with Instance Normalization Flows

Univariate Time Series Forecasting Via Interactive Learning.

State Causality and Adaptive Covariance Decomposition Based Time Series Forecasting.

Causal Discovery from Conditionally Stationary Time Series

Discovering Predictable Latent Factors for Time Series Forecasting

Probabilistic Learning of Multivariate Time Series with Temporal Irregularity

Fuzzy State-Driven Cross-Time Spatial Dependence Learning for Multivariate Time-Series Anomaly Detection

Identifiable Representation and Model Learning for Latent Dynamic Systems

Deep Frequency Derivative Learning for Non-stationary Time Series Forecasting

Latent state recognition by an enhanced hidden Markov model

Learning Temporally Causal Latent Processes from General Temporal Data.

DisenTS: Disentangled Channel Evolving Pattern Modeling for Multivariate Time Series Forecasting

Learning to infer weather states using partial observations