Abstract:The integration of observational data into numerical models, known as data assimilation (DA), is fundamental for making Numerical Weather Prediction (NWP) possible, with breathtaking success over the past 60 years (Bauer et al. 2015). Traditional DA methods, such as variational techniques and ensemble Kalman filters, are basic pillars of current NWP by incorporating diverse observational data. However, the emergence of artificial intelligence (AI) presents new opportunities for further improvements. AI-based approaches can emulate the complex computations of traditional NWP models at a reduced computational cost, offering the potential to speed up and improve analyses and forecasts dramatically (e.g. Pathak et al., 2022; Bi et al., 2023; Lam et al., 2023; Bouallegue et al., 2023). AI itself plays a growing role in optimization (e.g. Fan et al., 2024), which offers new possibilities also beyond model emulation. In this paper, we introduce a novel AI-based variational DA approach designed to replace classical methods of DA by leveraging deep learning techniques. Unlike previous hybrid approaches, our method integrates the DA process directly into a neural network, utilizing the variational DA framework. This innovative AI-based system, termed AI-Var, employs a neural network trained to minimize the variational cost function, enabling it to perform DA without relying on pre-existing analysis datasets. We present a proof-of-concept implementation of this approach, demonstrating its feasibility through a series of idealized and real-world test cases. Our results indicate that the AI-Var system can efficiently assimilate observations and produce accurate initial conditions for NWP, highlighting its potential to carry out the DA process in weather forecasting. This advancement paves the way for fully data-driven NWP systems, offering a significant leap forward in computational efficiency.

A cheap data assimilation approach for expensive numerical simulations

Adaptive Multifidelity Data Assimilation for Nonlinear Subsurface Flow Problems

A variational data assimilation approach for sparse velocity reference data in coarse RANS simulations through a corrective forcing term

Model and data reduction for data assimilation: Particle filters employing projected forecasts and data with application to a shallow water model

The Importance of Data Assimilation Components for Initial Conditions and Subsequent Error Growth

AI-based data assimilation: Learning the functional of analysis estimation

A new approach to data assimilation initialization problems with sparse data using multiple cost functions

Generative Data Assimilation of Sparse Weather Station Observations at Kilometer Scales

A reduced order with data assimilation model: Theory and practice

Data Assimilation using ERA5, ASOS, and the U-STN model for Weather Forecasting over the UK

Oceanic Data Assimilation Study in Northern Chile: Use of a 3DVAR Method

Fuxi-DA: A Generalized Deep Learning Data Assimilation Framework for Assimilating Satellite Observations

A Comparison of Two Nonlinear Data Assimilation Methods

Assessing the Feasibility of an NWP Satellite Data Assimilation System Entirely Based on AI Techniques

Neural Incremental Data Assimilation

A Multiscale Variational Data Assimilation Scheme: Formulation and Illustration

A Novel Deep Learning Approach for Data Assimilation of Complex Hydrological Systems

ADAF: An Artificial Intelligence Data Assimilation Framework for Weather Forecasting

An elastic framework for ensemble-based large-scale data assimilation

Data Assimilation Informed Model Structure Improvement (DAISI) for Robust Prediction Under Climate Change: Application to 201 Catchments in Southeastern Australia

An Efficient Computational Framework for Data Assimilation of Fractional-Order Dynamical System with Sparse Observations