Abstract:Skilful Machine Learned weather forecasts have challenged our approach to numerical weather prediction, demonstrating competitive performance compared to traditional physics-based approaches. Data-driven systems have been trained to forecast future weather by learning from long historical records of past weather such as the ECMWF ERA5. These datasets have been made freely available to the wider research community, including the commercial sector, which has been a major factor in the rapid rise of ML forecast systems and the levels of accuracy they have achieved. However, historical reanalyses used for training and real-time analyses used for initial conditions are produced by data assimilation, an optimal blending of observations with a physics-based forecast model. As such, many ML forecast systems have an implicit and unquantified dependence on the physics-based models they seek to challenge. Here we propose a new approach, training a neural network to predict future weather purely from historical observations with no dependence on reanalyses. We use raw observations to initialise a model of the atmosphere (in observation space) learned directly from the observations themselves. Forecasts of crucial weather parameters (such as surface temperature and wind) are obtained by predicting weather parameter observations (e.g. SYNOP surface data) at future times and arbitrary locations. We present preliminary results on forecasting observations 12-hours into the future. These already demonstrate successful learning of time evolutions of the physical processes captured in real observations. We argue that this new approach, by staying purely in observation space, avoids many of the challenges of traditional data assimilation, can exploit a wider range of observations and is readily expanded to simultaneous forecasting of the full Earth system (atmosphere, land, ocean and composition).

Decomposing weather forecasting into advection and convection with neural networks

Numerical Weather Forecasting using Convolutional-LSTM with Attention and Context Matcher Mechanisms

Hierarchical U-net with re-parameterization technique for spatio-temporal weather forecasting

Deep Learning for Weather Forecasting: A CNN-LSTM Hybrid Model for Predicting Historical Temperature Data

Short-term precipitation prediction using deep learning

Precipitation Forecasting via Multi-Scale Deconstructed ConvLSTM

Weather Forecasting Using Ensemble of Spatial-Temporal Attention Network and Multi-Layer Perceptron

FuXi Weather: an End-to-end Machine Learning Weather Data Assimilation and Forecasting System

Forecasting the Future with Future Technologies: Advancements in Large Meteorological Models

Exploring the Use of Machine Learning Weather Models in Data Assimilation

Accurate medium-range global weather forecasting with 3D neural networks

Hybrid wind speed forecasting model based on multivariate data secondary decomposition approach and deep learning algorithm with attention mechanism

Multi-Information Aggregation and Estrangement HyperGraph Convolutional Networks for Spatiotemporal Weather Forecasting

A Structured Graph Neural Network for Improving the Numerical Weather Prediction of Rainfall

Data-Driven Weather Forecasting and Climate Modeling from the Perspective of Development

Data driven weather forecasts trained and initialised directly from observations

Efficient Subseasonal Weather Forecast using Teleconnection-informed Transformers

Decomposition-based wind speed forecasting model using causal convolutional network and attention mechanism

Convolutional Neural Network For Short Term Fog Forecasting Based On Meteorological Elements

Spatiotemporal wind speed forecasting using conditional local convolution and multidimensional meteorology features

A Method Based on Deep Learning for Severe Convective Weather Forecast: CNN-BiLSTM-AM (Version 1.0)