AI Foundation Models for Weather and Climate: Applications, Design, and Implementation

S. Karthik Mukkavilli,Daniel Salles Civitarese,Johannes Schmude,Johannes Jakubik,Anne Jones,Nam Nguyen,Christopher Phillips,Sujit Roy,Shraddha Singh,Campbell Watson,Raghu Ganti,Hendrik Hamann,Udaysankar Nair,Rahul Ramachandran,Kommy Weldemariam

2023-09-20

Abstract:Machine learning and deep learning methods have been widely explored in understanding the chaotic behavior of the atmosphere and furthering weather forecasting. There has been increasing interest from technology companies, government institutions, and meteorological agencies in building digital twins of the Earth. Recent approaches using transformers, physics-informed machine learning, and graph neural networks have demonstrated state-of-the-art performance on relatively narrow spatiotemporal scales and specific tasks. With the recent success of generative artificial intelligence (AI) using pre-trained transformers for language modeling and vision with prompt engineering and fine-tuning, we are now moving towards generalizable AI. In particular, we are witnessing the rise of AI foundation models that can perform competitively on multiple domain-specific downstream tasks. Despite this progress, we are still in the nascent stages of a generalizable AI model for global Earth system models, regional climate models, and mesoscale weather models. Here, we review current state-of-the-art AI approaches, primarily from transformer and operator learning literature in the context of meteorology. We provide our perspective on criteria for success towards a family of foundation models for nowcasting and forecasting weather and climate predictions. We also discuss how such models can perform competitively on downstream tasks such as downscaling (super-resolution), identifying conditions conducive to the occurrence of wildfires, and predicting consequential meteorological phenomena across various spatiotemporal scales such as hurricanes and atmospheric rivers. In particular, we examine current AI methodologies and contend they have matured enough to design and implement a weather foundation model.

Machine Learning,Artificial Intelligence,Atmospheric and Oceanic Physics

What problem does this paper attempt to address?

The paper primarily explores how to utilize AI foundation models to address several key issues in weather and climate prediction. Specifically:

1. **Weather Forecasting**: Achieving rapid short-term weather forecasts (such as precipitation) through machine learning techniques, particularly in image processing. These methods have significant speed advantages over traditional numerical weather prediction (NWP) models, but currently have limitations in spatial resolution and long-term forecast accuracy.

2. **Model Fusion and Post-Processing**: Improving the output of existing NWP models using machine learning techniques to enhance forecast accuracy. This approach typically involves fusing or post-processing the outputs of multiple models, helping to correct biases in the original models.

3. **Downscaling (Super-Resolution)**: Converting low-resolution weather data into high-resolution data to meet more detailed meteorological needs. This requires addressing differences between various NWP models and creating consistent datasets for training.

4. **Parameterization**: Investigating how to use machine learning methods to replace traditional parameterization schemes in NWP models to better represent sub-grid scale processes. For example, training deep learning models to simulate cloud behavior, thereby enhancing the accuracy of climate models.

5. **Data Assimilation**: Exploring how to apply deep learning techniques to improve the data assimilation process, thereby increasing the accuracy of initial state estimates and constraining the development trends of future states.

6. **Detection and Prediction of Weather Patterns**: Using deep learning techniques to automatically identify extreme weather events (such as hurricanes, tornadoes, etc.) to improve the assessment of future climate change risks. This method can avoid the reliance on specific variables and thresholds in traditional methods and better handle small sample datasets.

In summary, this paper aims to address multiple specific tasks in the field of weather and climate through the construction of general AI foundation models, including but not limited to short-term forecasting, model fusion, downscaling, parameterization improvements, and the detection and prediction of extreme weather events.