Abstract:Learning identifiable representations and models from low-level observations is useful for an intelligent spacecraft to reliability finish downstream tasks. For temporal observations, to ensure that the data generating process is provably inverted, most existing works either assume the noise variables in the dynamic mechanisms are (conditionally) independent, or require interventions which can directly affect each latent variable. However, in practice, the relationship between the exogenous inputs/interventions and the latent variables may follow some complex deterministic mechanisms. In this work, we study the problem of identifiable representation and model learning for latent dynamic systems. The key idea is that we use an inductive bias inspired by controllable canonical forms, which is invariant, sparse, and input dependent by definition. We prove that, for linear or affine nonlinear latent dynamic systems, it is possible to identify the representations up to scaling and determine the models up to some simple transformations. The results have potential to provide some theoretical guarantees for developing more trustworthy decision-making and control methods for intelligent spacecrafts.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to learn identifiable representations and models from low - level observational data so that intelligent spacecraft can reliably complete downstream tasks. Specifically, the paper focuses on the problem of representation and model learning in latent dynamic systems, especially in cases where there are high - order deterministic mechanisms in these systems. ### Problem Background When performing tasks in complex environments, intelligent spacecraft may encounter low - level observational data such as images or outputs of pre - trained neural networks. These observational data themselves usually have no physical meaning, and the relationships between them are usually highly nonlinear and unknown. However, there may be some high - level latent variables that can represent these observational data, and there are some invariant mechanisms that can describe the relationships between these latent variables and exogenous inputs. ### Limitations of Existing Methods Most existing works ensure that the data generation process is reversible by assuming that the noise variables in the dynamic mechanism are (conditionally) independent or by directly intervening in each latent variable. But in practical applications, the relationships between exogenous inputs/interventions and latent variables may follow some complex deterministic mechanisms, which makes causal relationships exist between variables. Therefore, it is necessary to learn representations and mechanisms simultaneously. ### Core Contributions of the Paper 1. **Learning of Identifiable Representations and Models**: - The paper shows that for linear or affine - nonlinear latent dynamic systems, representations (up to scaling transformations) can be identified and models (up to simple transformations) can be determined by using the inductive bias inspired by the controllable canonical form. - This is the first result of identifiable representation learning for deterministic high - order latent dynamic systems. 2. **Results of Representation Learning**: - For affine - nonlinear latent dynamic systems, the representation can be identified up to the level of scaling transformation even in the presence of deterministic mechanisms. 3. **Results of Model Learning**: - For single - input linear systems, the coefficients in the system matrix can be identified as the true values. - For multi - input linear systems and affine - nonlinear systems, the coefficients or coefficient functions can be identified up to simple transformations that do not affect one - step prediction. ### Method Overview The paper proposes a representation and model learning method based on the controllable canonical form, which can capture the indirect influence of exogenous inputs and is applicable to high - order dynamic systems. Through this method, the representation and model of the latent dynamic system can be learned under the premise of ensuring the sparsity of the representation and the input - dependence. ### Conclusion This research provides theoretical support for the development of more trustworthy decision - making and control methods, especially for the task execution of intelligent spacecraft in rich - observation environments.

Identifiable Representation and Model Learning for Latent Dynamic Systems

Identifiability and Identification of Switching Dynamical Networks: A Data-Based Approach

Tracking control of latent dynamic systems with application to spacecraft attitude control

Identifying latent state transition in non-linear dynamical systems

Unsupervised Learning of Hybrid Latent Dynamics: A Learn-to-Identify Framework

Analyzing and Improving Supervised Nonlinear Dynamical Probabilistic Latent Variable Model for Inferential Sensors

On the Identifiability of Switching Dynamical Systems

Identifying Latent Dynamic Components in Biological Systems

Identifiable Latent Neural Causal Models

On the Identification of Temporally Causal Representation with Instantaneous Dependence

Marrying Causal Representation Learning with Dynamical Systems for Science

Learning Latent Causal Dynamics

Dynamically Meaningful Latent Representations of Dynamical Systems

Deep Generative Modeling for Identification of Noisy, Non-Stationary Dynamical Systems

Temporally Disentangled Representation Learning under Unknown Nonstationarity

Learning Latent Dynamics via Invariant Decomposition and (Spatio-)Temporal Transformers

Meta-State-Space Learning: An Identification Approach for Stochastic Dynamical Systems

From latent dynamics to meaningful representations

Identifiable Latent Polynomial Causal Models Through the Lens of Change

Continual Learning of Nonlinear Independent Representations