Abstract:Temporally causal representation learning aims to identify the latent causal process from time series observations, but most methods require the assumption that the latent causal processes do not have instantaneous relations. Although some recent methods achieve identifiability in the instantaneous causality case, they require either interventions on the latent variables or grouping of the observations, which are in general difficult to obtain in real-world scenarios. To fill this gap, we propose an \textbf{ID}entification framework for instantane\textbf{O}us \textbf{L}atent dynamics (\textbf{IDOL}) by imposing a sparse influence constraint that the latent causal processes have sparse time-delayed and instantaneous relations. Specifically, we establish identifiability results of the latent causal process based on sufficient variability and the sparse influence constraint by employing contextual information of time series data. Based on these theories, we incorporate a temporally variational inference architecture to estimate the latent variables and a gradient-based sparsity regularization to identify the latent causal process. Experimental results on simulation datasets illustrate that our method can identify the latent causal process. Furthermore, evaluations on multiple human motion forecasting benchmarks with instantaneous dependencies indicate the effectiveness of our method in real-world settings.

What problem does this paper attempt to address?

The paper is primarily dedicated to addressing the problem of identifying latent causal processes in time series data, especially in the presence of instantaneous dependencies. Specifically, the paper addresses the following issues: 1. **Challenging Assumptions**: Most existing methods require that the latent causal processes do not have instantaneous relationships (i.e., latent variables are independent at their previous timestamps), which is a strong assumption in practical applications. 2. **Handling Instantaneous Dependencies**: While some recent methods can handle instantaneous causal relationships, these methods typically require interventions on latent variables or grouping of observed variables, both of which are often difficult to achieve in real-world applications. ### Research Objectives The main objective of the paper is to propose a new framework that can identify latent causal processes with instantaneous dependencies in time series without relying on interventions or grouping. To this end, the authors propose a framework named IDOL, which is based on the following key points: - **Sparse Influence Constraint**: The model is built by assuming that the instantaneous and delayed dependencies in the latent causal processes are sparse. - **Leveraging Contextual Information of Time Series Data**: The model enhances its identification capability by utilizing the variation characteristics of time series data across multiple timestamps (i.e., "sufficient variability"). - **Theoretical Foundation**: The paper establishes theoretical results based on the sufficient variability assumption and sparse influence constraint, demonstrating the identifiability of the latent causal processes. - **Model Design**: A model combining a temporal variational inference architecture and gradient-based sparse regularization is designed to estimate latent variables and identify latent causal processes. ### Method Overview - **Theoretical Contributions**: - Proposes the relationship between latent variables and estimated variables in time series based on contextual information (see Theorem 1). - Establishes identifiability results for latent variables under the sparse causal influence assumption (see Theorem 2). - **Model Architecture**: - Uses an encoder-decoder structure to extract latent variables and reconstruct observations. - Introduces a prior network to estimate the prior distribution of latent variables and ensure the independence of noise terms. - Applies gradient-based sparse regularization to promote sparsity in the latent causal processes. ### Experimental Validation - **Simulated Data Experiments**: The proposed method's effectiveness and robustness are validated through datasets with different levels of sparsity. - **Real Data Experiments**: The method is evaluated on two real human motion prediction datasets, demonstrating its effectiveness. In summary, this paper aims to address the problem of identifying latent causal processes with instantaneous dependencies by introducing the sparsity assumption and leveraging the characteristics of time series data, thereby providing a more practical and general approach for time series analysis.

On the Identification of Temporally Causal Representation with Instantaneous Dependence

On the identification of temporally causal representation with instantaneous dependence

Temporally Disentangled Representation Learning under Unknown Nonstationarity

Causal Representation Learning for Instantaneous and Temporal Effects in Interactive Systems

CaRiNG: Learning Temporal Causal Representation under Non-Invertible Generation Process

Causal Representation Learning Made Identifiable by Grouping of Observational Variables

Causal Representation Learning in Temporal Data via Single-Parent Decoding

Identifiable Representation and Model Learning for Latent Dynamic Systems

Identifiable Latent Neural Causal Models

Identifying Weight-Variant Latent Causal Models

Unifying Causal Representation Learning with the Invariance Principle

Learning Temporally Causal Latent Processes from General Temporal Data.

Continual Learning of Nonlinear Independent Representations

Learning Latent Causal Dynamics

When and How: Learning Identifiable Latent States for Nonstationary Time Series Forecasting

Learning Causal Representations from General Environments: Identifiability and Intrinsic Ambiguity

On the Recoverability of Causal Relations from Temporally Aggregated I.I.D. Data

General Identifiability and Achievability for Causal Representation Learning

Identifiable Causal Representation Learning: Unsupervised, Multi-View, and Multi-Environment

Dynamical causality under invisible confounders

Causal Temporal Representation Learning with Nonstationary Sparse Transition