Abstract:Skilful Machine Learned weather forecasts have challenged our approach to numerical weather prediction, demonstrating competitive performance compared to traditional physics-based approaches. Data-driven systems have been trained to forecast future weather by learning from long historical records of past weather such as the ECMWF ERA5. These datasets have been made freely available to the wider research community, including the commercial sector, which has been a major factor in the rapid rise of ML forecast systems and the levels of accuracy they have achieved. However, historical reanalyses used for training and real-time analyses used for initial conditions are produced by data assimilation, an optimal blending of observations with a physics-based forecast model. As such, many ML forecast systems have an implicit and unquantified dependence on the physics-based models they seek to challenge. Here we propose a new approach, training a neural network to predict future weather purely from historical observations with no dependence on reanalyses. We use raw observations to initialise a model of the atmosphere (in observation space) learned directly from the observations themselves. Forecasts of crucial weather parameters (such as surface temperature and wind) are obtained by predicting weather parameter observations (e.g. SYNOP surface data) at future times and arbitrary locations. We present preliminary results on forecasting observations 12-hours into the future. These already demonstrate successful learning of time evolutions of the physical processes captured in real observations. We argue that this new approach, by staying purely in observation space, avoids many of the challenges of traditional data assimilation, can exploit a wider range of observations and is readily expanded to simultaneous forecasting of the full Earth system (atmosphere, land, ocean and composition).

Leveraging data-driven weather models for improving numerical weather prediction skill through large-scale spectral nudging

The rise of data-driven weather forecasting

Forecasting the Future with Future Technologies: Advancements in Large Meteorological Models

Improving Typhoon Predictions by Integrating Data-Driven Machine Learning Models with Physics Models Based on the Spectral Nudging and Data Assimilation

Learning skillful medium-range global weather forecasting

Validating Deep-Learning Weather Forecast Models on Recent High-Impact Extreme Events

GraphCast: Learning skillful medium-range global weather forecasting

Data-Driven Weather Forecasting and Climate Modeling from the Perspective of Development

Exploring the Use of Machine Learning Weather Models in Data Assimilation

A Scalable Real-Time Data Assimilation Framework for Predicting Turbulent Atmosphere Dynamics

Lightning-Fast Convective Outlooks: Predicting Severe Convective Environments with Global AI-based Weather Models

Accelerating Community-Wide Evaluation of AI Models for Global Weather Prediction by Facilitating Access to Model Output

Hybrid physics-AI outperforms numerical weather prediction for extreme precipitation nowcasting

Data driven weather forecasts trained and initialised directly from observations

Short-range forecasts of global precipitation using deep learning-augmented numerical weather prediction

Robustness of AI-based weather forecasts in a changing climate

Generalizing Weather Forecast to Fine-grained Temporal Scales via Physics-AI Hybrid Modeling

FourCastNet: Accelerating Global High-Resolution Weather Forecasting using Adaptive Fourier Neural Operators

Do AI models produce better weather forecasts than physics-based models? A quantitative evaluation case study of Storm Ciarán

FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators