Abstract:Variational Data Assimilation (DA) has been broadly used in engineering problems for field reconstruction and prediction by performing a weighted combination of multiple sources of noisy data. In recent years, the integration of deep learning (DL) techniques in DA has shown promise in improving the efficiency and accuracy in high-dimensional dynamical systems. Nevertheless, existing deep DA approaches face difficulties in dealing with unstructured observation data, especially when the placement and number of sensors are dynamic over time. We introduce a novel variational DA scheme, named Voronoi-tessellation Inverse operator for VariatIonal Data assimilation (VIVID), that incorporates a DL inverse operator into the assimilation objective function. By leveraging the capabilities of the Voronoi-tessellation and convolutional neural networks, VIVID is adept at handling sparse, unstructured, and time-varying sensor data. Furthermore, the incorporation of the DL inverse operator establishes a direct link between observation and state space, leading to a reduction in the number of minimization steps required for DA. Additionally, VIVID can be seamlessly integrated with Proper Orthogonal Decomposition (POD) to develop an end-to-end reduced-order DA scheme, which can further expedite field reconstruction. Numerical experiments in a fluid dynamics system demonstrate that VIVID can significantly outperform existing DA and DL algorithms. The robustness of VIVID is also accessed through the application of various levels of prior error, the utilization of varying numbers of sensors, and the misspecification of error covariance in DA.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to improve the efficiency and accuracy of Variational Data Assimilation (DA) when dealing with sparse, unstructured, and time - varying sensor data. Specifically, the existing deep - learning - assisted data assimilation methods face difficulties in dealing with the situation where the sensor locations and numbers change dynamically. Therefore, this paper proposes a new deep - learning - assisted data assimilation scheme - Voronoi - tessellation Inverse operator for VariatIonal Data assimilation (VIVID) - to address these issues. ### Main problems and challenges 1. **Sparse and unstructured observation data**: - The existing deep data assimilation methods have difficulty in handling irregularly distributed observation data, especially when the sensor locations and numbers change over time. 2. **Computational efficiency**: - Traditional data assimilation methods require a large number of iterations to solve complex optimization problems, and the computational cost is very high, especially in high - dimensional dynamic systems. 3. **Limitations of existing methods**: - Although machine - learning methods show potential in improving prediction efficiency and accuracy, they can usually only handle structured observation data (such as square grids) and perform better when the number of sensors is fixed. This limits their wide use in practical applications. ### Innovations of VIVID - **Introduction of Voronoi tessellation**: By using Voronoi tessellation to divide the observation space into multiple regions, it can effectively handle sparse and unstructured observation data. - **Combination with Convolutional Neural Network (CNN)**: Utilize CNN to map the segmented observation data to the state space, which reduces the dependence on initial conditions and accelerates the optimization process. - **End - to - end optimization framework**: VIVID is an end - to - end optimization framework that can simultaneously minimize background error, inverse error, and observation error, improving the overall assimilation accuracy. - **Integration of Reduced Order Model (ROM)**: Through Proper Orthogonal Decomposition (POD) for reduced - order modeling, the computational efficiency is further improved. ### Experimental verification The author verifies the performance of VIVID in a two - dimensional fluid dynamics system through numerical experiments. The results show that VIVID is significantly superior to traditional variational data assimilation methods and existing deep - learning field reconstruction methods (such as VCNN) in both reconstruction accuracy and computational efficiency. In addition, the robustness of VIVID under different noise levels and assumptions is also evaluated. In summary, this paper aims to address the deficiencies of existing deep data assimilation methods in dealing with sparse, unstructured, and time - varying sensor data, and proposes a novel and efficient solution, VIVID.

Efficient deep data assimilation with sparse observations and time-varying sensors

Deep Learning Augmented Data Assimilation: Reconstructing Missing Information with Convolutional Autoencoders

A Four‐Dimensional Variational Constrained Neural Network‐Based Data Assimilation Method

AI-based data assimilation: Learning the functional of analysis estimation

A Novel Deep Learning Approach for Data Assimilation of Complex Hydrological Systems

Advancing neural network-based data assimilation for large-scale spatiotemporal systems with sparse observations

3D‐Var data assimilation using a variational autoencoder

Combined Optimization of Dynamics and Assimilation with End-to-End Learning on Sparse Observations

Deep Learning for Latent Space Data Assimilation LSDA in Subsurface Flow Systems

Learning Dynamical Characteristics with Neural Operators for Data Assimilation

Deep Learning for Latent Space Data Assimilation in Subsurface Flow Systems

3D-Var Data Assimilation using a Variational Autoencoder

VAE-Var: Variational-Autoencoder-Enhanced Variational Assimilation

Analyzing and Improving Supervised Nonlinear Dynamical Probabilistic Latent Variable Model for Inferential Sensors

Variational assimilation of sparse time-averaged data for efficient adjoint-based optimization of unsteady RANS simulations

Generalised Latent Assimilation in Heterogeneous Reduced Spaces with Machine Learning Surrogate Models

$Φ$-DVAE: Physics-Informed Dynamical Variational Autoencoders for Unstructured Data Assimilation

Neural Incremental Data Assimilation

A variational data assimilation approach for sparse velocity reference data in coarse RANS simulations through a corrective forcing term

Convergent least-squares optimization methods for variational data assimilation

A scalable space-time domain decomposition approach for solving large-scale nonlinear regularized inverse ill-posed problems in 4D variational data assimilation