On‐line machine‐learning forecast uncertainty estimation for sequential data assimilation

Maximiliano A. Sacco,Manuel Pulido,Juan J. Ruiz,Pierre Tandeo
DOI: https://doi.org/10.1002/qj.4743
2024-05-10
Quarterly Journal of the Royal Meteorological Society
Abstract:Artificial neural network training scheme. The training error is determined by the Frobenius norm between ∑ ̃k and the training matrix εkf(εkf)T , which is estimated from the approximated forecast error. Quantifying forecast uncertainty is a key aspect of state‐of‐the‐art numerical weather prediction and data assimilation systems. Ensemble‐based data assimilation systems incorporate state‐dependent uncertainty quantification based on multiple model integrations. However, this approach is demanding in terms of computations and development. In this work, a machine‐learning method is presented based on convolutional neural networks that estimates the state‐dependent forecast uncertainty represented by the forecast error covariance matrix using a single dynamical model integration. This is achieved by the use of a loss function that takes into account the fact that the forecast errors are heteroscedastic. The performance of this approach is examined within a hybrid data assimilation method that combines a Kalman‐like analysis update and the machine‐learning‐based estimation of a state‐dependent forecast error covariance matrix. Observing system simulation experiments are conducted using the Lorenz'96 model as a proof‐of‐concept. The promising results show that the machine‐learning method is able to predict precise values of the forecast covariance matrix in relatively high‐dimensional states. Moreover, the hybrid data assimilation method shows similar performance to the ensemble Kalman filter, outperforming it when the ensembles are relatively small.
meteorology & atmospheric sciences
What problem does this paper attempt to address?