Abstract:Continuous-time (CT) modeling has proven to provide improved sample efficiency and interpretability in learning the dynamical behavior of physical systems compared to discrete-time (DT) models. However, even with numerous recent developments, the CT nonlinear state-space (NL-SS) model identification problem remains to be solved in full, considering common experimental aspects such as the presence of external inputs, measurement noise, latent states, and general robustness. This paper presents a novel estimation method that addresses all these aspects and that can obtain state-of-the-art results on multiple benchmarks with compact fully connected neural networks capturing the CT dynamics. The proposed estimation method called the subspace encoder approach (SUBNET) ascertains these results by efficiently approximating the complete simulation loss by evaluating short simulations on subsections of the data, by using an encoder function to estimate the initial state for each subsection and a novel state-derivative normalization to ensure stability and good numerical conditioning of the training process. We prove that the use of subsections increases cost function smoothness together with the necessary requirements for the existence of the encoder function and we show that the proposed state-derivative normalization is essential for reliable estimation of CT NL-SS models.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve the identification problem of continuous - time (CT) nonlinear state - space (NL - SS) models. Specifically, the paper focuses on how to estimate CT NL - SS models from data in the presence of external inputs, measurement noise, hidden states, and requirements for robustness. These problems are very common in practical applications, such as the modeling and control of physical systems. #### Main problem description 1. **Continuous - time models vs. discrete - time models**: - The paper points out that although discrete - time (DT) models are easier to obtain from data, CT models can use samples more effectively and have more advantages in explaining the dynamic behavior of physical systems. - CT models are more suitable for handling irregular sampling or missing data situations and are more intuitive in control tasks. 2. **Challenges and difficulties**: - **External input**: The system is usually affected by external signals. - **Measurement noise**: Actual measurement data often contains noise. - **Hidden state**: There may be states within the system that cannot be directly observed. - **Unknown output function**: The output function of the system may be unknown. - **Long - term prediction**: The model is required to have accurate long - term prediction ability. - **Computational cost**: It is necessary to reduce the computational complexity while ensuring performance. 3. **Deficiencies of existing methods**: - Most of the current methods only solve part of the above problems, and there is no comprehensive solution to deal with all these challenges simultaneously. #### Proposed method To solve these problems, the paper proposes a new estimation method, called the **Subspace Encoder Approach (SUBNET)**. The main features of this method include: - **Short - segment simulation**: Reduce the computational complexity by dividing the data set into multiple overlapping short segments. - **Encoder function**: Use the encoder function to estimate the initial state of each segment, thus avoiding taking the initial state as a free optimization parameter. - **State derivative normalization**: Introduce a new state derivative normalization term to ensure the stability of the training process and good numerical conditions. Through these improvements, the SUBNET method can achieve state - of - the - art results in multiple benchmark tests and perform well in terms of computational efficiency and robustness. #### Summary of mathematical formulas The key formulas in the paper include: - System dynamics equation: \[ \dot{x}(t)=f(x(t), u(t)) \] where \(x(t)\in\mathbb{R}^{n_x}\) is the system state variable and \(u(t)\in\mathbb{R}^{n_u}\) is the input. - Output equation: \[ y_k = h(x_s(k\Delta t), u_k)+w_k \] where \(y_k\in\mathbb{R}^{n_y}\) is the output and \(w_k\) is zero - mean white noise. - Loss function minimization problem: \[ \min_{\theta, x(0)}\frac{1}{N}\sum_{k = 0}^{N - 1}\|y(k\Delta t)-\hat{y}(k\Delta t)\|^2_2 \] where \(\hat{y}(t)=h_\theta(x(t))\), \(\dot{x}(t)=f_\theta(x(t), u(t))\). - Subspace encoder optimization problem: \[ \minimize_{\theta}\frac{1}{N - T-\max(n_a,n_b)+1}\sum_{n=\max(n_a,n_b)}^{N - T}\frac{1}{T}\sum_{k =

Continuous-time identification of dynamic state-space models by deep subspace encoding

Deep Subspace Encoders for Nonlinear System Identification

Initialization Approach for Nonlinear State-Space Identification via the Subspace Encoder Approach

Closed-Loop Subspace Identification Based on Kpls

Identifiability and Identification of Switching Dynamical Networks: A Data-Based Approach

A new closed-loop identification technique for step-response model based on subspace approach

Subspace Identification for Continuous-Time Errors-in-variables Model from Sampled Data

Closed-loop Subspace Identification for Continuous-time Models

Investigating the Surrogate Modeling Capabilities of Continuous Time Echo State Networks

Subspace Identification of Large-Scale 1D Homogeneous Networks

State Derivative Normalization for Continuous-Time Deep Neural Networks

Deep Neural Network-Embedded Stochastic Nonlinear State-Space Models and Their Applications to Process Monitoring

Subspace-based Continuous-time Model Identification Using Nuclear Norm Minimization

Recursive Closed-Loop Subspace Identification Based on Predictor Markov Parameters

Differential Neural Network Identifier for Dynamical Systems With Time-Varying State Constraints

Deep State Space Models for Nonlinear System Identification

Koopman dynamic-oriented deep learning for invariant subspace identification and full-state prediction of complex systems

Meta-State-Space Learning: An Identification Approach for Stochastic Dynamical Systems

Modeling of forced-vibration systems using continuous-time state-space neural network

Unifying Theorems for Subspace Identification and Dynamic Mode Decomposition

Expectation maximization based sparse identification of cyberphysical system