Evaluating the transferability potential of deep learning models for climate downscaling

Ayush Prasad,Paula Harder,Qidong Yang,Prasanna Sattegeri,Daniela Szwarcman,Campbell Watson,David Rolnick

2024-07-17

Abstract:Climate downscaling, the process of generating high-resolution climate data from low-resolution simulations, is essential for understanding and adapting to climate change at regional and local scales. Deep learning approaches have proven useful in tackling this problem. However, existing studies usually focus on training models for one specific task, location and variable, which are therefore limited in their generalizability and transferability. In this paper, we evaluate the efficacy of training deep learning downscaling models on multiple diverse climate datasets to learn more robust and transferable representations. We evaluate the effectiveness of architectures zero-shot transferability using CNNs, Fourier Neural Operators (FNOs), and vision Transformers (ViTs). We assess the spatial, variable, and product transferability of downscaling models experimentally, to understand the generalizability of these different architecture types.

Machine Learning

What problem does this paper attempt to address?

The paper aims to address the transferability issue of deep learning models in the process of climate downscaling. Specifically:

1. **Problem Background**: Climate downscaling refers to generating high-resolution climate data from low-resolution climate simulation data, which is crucial for understanding and adapting to climate change at regional and local scales. Although deep learning methods have shown excellent performance in this field, existing studies typically focus on training models for a single task, location, and variable, resulting in limited generalization and transferability of these models.

2. **Research Objective**: This paper evaluates whether deep learning models trained on multiple diverse climate datasets learn more robust and transferable representations. The authors focus on the zero-shot transferability of Convolutional Neural Networks (CNNs), Fourier Neural Operators (FNOs), and Vision Transformers (ViTs), and experimentally assess their transfer performance across space, variables, and data products.

3. **Main Findings**:
- In spatial transferability experiments, all three models outperformed the bicubic interpolation baseline, with the CNN-ViT hybrid model performing the best.
- In variable transferability experiments, the FNO model performed the best for unseen variables (e.g., downward longwave radiation flux).
- For transferability between different data products, the CNN-ViT model again performed the best.
- In practical tests with low-resolution and high-resolution data pairs (NorESM dataset), after fine-tuning, the CNN-ViT model and the FNO model performed excellently.

In summary, this study aims to explore how pre-training on multiple datasets combined with fine-tuning can enhance the generalization and transferability of deep learning models in climate downscaling tasks.