GPTCast: a weather language model for precipitation nowcasting

Gabriele Franch,Elena Tomasi,Rishabh Wanjari,Virginia Poli,Chiara Cardinali,Pier Paolo Alberoni,Marco Cristoforetti

2024-09-24

Abstract:This work introduces GPTCast, a generative deep-learning method for ensemble nowcast of radar-based precipitation, inspired by advancements in large language models (LLMs). We employ a GPT model as a forecaster to learn spatiotemporal precipitation dynamics using tokenized radar images. The tokenizer is based on a Quantized Variational Autoencoder featuring a novel reconstruction loss tailored for the skewed distribution of precipitation that promotes faithful reconstruction of high rainfall rates. The approach produces realistic ensemble forecasts and provides probabilistic outputs with accurate uncertainty estimation. The model is trained without resorting to randomness, all variability is learned solely from the data and exposed by model at inference for ensemble generation. We train and test GPTCast using a 6-year radar dataset over the Emilia-Romagna region in Northern Italy, showing superior results compared to state-of-the-art ensemble extrapolation methods.

Machine Learning,Atmospheric and Oceanic Physics

What problem does this paper attempt to address?

The paper aims to address the problem of short-term precipitation forecasting (nowcasting), particularly for early warnings of water-related disasters such as floods and landslides caused by sudden precipitation. Traditional methods like the Lagrangian extrapolation have limitations in handling convective precipitation, while Numerical Weather Prediction (NWP) models can predict the probability and intensity of precipitation over larger areas but have limited accuracy on smaller geographic and temporal scales. Therefore, this paper proposes a new method called GPTCast, which is a generative deep learning approach that applies large language model (LLM) techniques from the natural language processing domain to the tokenization and spatiotemporal prediction of radar precipitation images.

Specifically, GPTCast utilizes a tokenizer based on Vector Quantized Variational Autoencoder (VQ-VAE) to learn how to convert radar images into discrete representations and employs an improved loss function (MWAE) to better reconstruct high rainfall rate scenarios. Then, the method uses an autoregressive model similar to the GPT-2 architecture for spatiotemporal forecasting, thereby generating reliable ensemble forecasts and providing probabilistic outputs with accurate uncertainty estimates. In this way, GPTCast can learn all variations from the data without introducing randomness and generate ensemble forecasts during inference. Experimental results show that GPTCast outperforms existing state-of-the-art ensemble extrapolation methods on a 6-year radar dataset from the Emilia-Romagna region in Italy.