Abstract:This study explores integrating reinforcement learning (RL) with idealised climate models to address key parameterisation challenges in climate science. Current climate models rely on complex mathematical parameterisations to represent sub-grid scale processes, which can introduce substantial uncertainties. RL offers capabilities to enhance these parameterisation schemes, including direct interaction, handling sparse or delayed feedback, continuous online learning, and long-term optimisation. We evaluate the performance of eight RL algorithms on two idealised environments: one for temperature bias correction, another for radiative-convective equilibrium (RCE) imitating real-world computational constraints. Results show different RL approaches excel in different climate scenarios with exploration algorithms performing better in bias correction, while exploitation algorithms proving more effective for RCE. These findings support the potential of RL-based parameterisation schemes to be integrated into global climate models, improving accuracy and efficiency in capturing complex climate dynamics. Overall, this work represents an important first step towards leveraging RL to enhance climate model accuracy, critical for improving climate understanding and predictions. Code accessible at <a class="link-external link-https" href="https://github.com/p3jitnath/climate-rl" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

This paper attempts to address the challenges of parameterisation schemes in climate models, especially how to improve the representation of sub - grid scale processes in numerical weather and climate models. Current climate models rely on complex mathematical parameterisations to represent these processes, which may introduce significant uncertainties. By combining Reinforcement Learning (RL) with idealised climate models, researchers hope to enhance these parameterisation schemes, thereby improving the accuracy and efficiency of climate models. ### Main problems 1. **Complexity of sub - grid scale processes**: Existing climate models have limitations in dealing with sub - grid scale processes, resulting in less accurate prediction results. 2. **Optimisation of parameterisation schemes**: Traditional parameterisation methods are difficult to adjust dynamically and cannot effectively cope with the complexity and diversity of climate change. 3. **Long - term optimisation and sparse feedback**: Climate models need to handle data over long time spans, and the feedback may be sparse or delayed, which poses a challenge to traditional machine - learning methods. ### Solutions The paper proposes using Reinforcement Learning algorithms to improve the parameterisation schemes of climate models, specifically including: - **Direct interaction**: RL can interact with the environment in real - time and adjust parameters dynamically. - **Handling sparse or delayed feedback**: RL is good at learning from sparse or delayed feedback and is suitable for the characteristics of climate data. - **Continuous online learning**: RL can keep learning and optimising during the model running process without retraining the entire model. - **Long - term optimisation**: RL can focus on maximising long - term rewards, which helps to capture long - term climate patterns and trends. ### Experimental setup To evaluate the effectiveness of RL algorithms, researchers designed two idealised environments: 1. **SimpleClimateBiasCorrectionEnv (temperature bias correction environment)**: Used to test the performance of RL algorithms in correcting temperature biases. 2. **RadiativeConvectiveModelEnv (radiative - convective equilibrium environment)**: Simulates the computational constraints in the real world and tests the performance of RL algorithms in more complex climate scenarios. ### Results The experimental results show that different RL algorithms perform well in different environments: - In the temperature bias correction environment, off - policy algorithms (such as DDPG, TD3, TQC) perform best. - In the radiative - convective equilibrium environment, trust - region - based policies (such as DPG, PPO, TRPO) are more effective. These results indicate that RL has great potential to improve the accuracy and efficiency of climate models by dynamically adjusting parameterisation schemes.

RAIN: Reinforcement Algorithms for Improving Numerical Weather and Climate Models

Optimizing Heat Alert Issuance with Reinforcement Learning

Crafting desirable climate trajectories with RL explored socio-environmental simulations

Deep Reinforcement Learning with Uncertain Data for Real-Time Stormwater System Control and Flood Mitigation

Climate Adaptation with Reinforcement Learning: Experiments with Flooding and Transportation in Copenhagen

Reinforcement Learning for Sociohydrology

Reducing climate model biases by exploring parameter space with large ensembles of climate model simulations and statistical emulation

Flood mitigation in coastal urban catchments using real-time stormwater infrastructure control and reinforcement learning

ClimateLearn: Benchmarking Machine Learning for Weather and Climate Modeling

Deep Reinforcement Learning for Conservation Decisions

Active Reinforcement Learning for Robust Building Control

Reinforcement Learning-based Model Predictive Control for Greenhouse Climate Control

ACE: A fast, skillful learned global atmospheric model for climate prediction

Reinforcement Learning Versus Model Predictive Control on Greenhouse Climate Control

Neural Network Parameterization of Subgrid‐Scale Physics From a Realistic Geography Global Storm‐Resolving Simulation

AI Foundation Models for Weather and Climate: Applications, Design, and Implementation

Machine learning for numerical weather and climate modelling: a review

A Non‐Intrusive Machine Learning Framework for Debiasing Long‐Time Coarse Resolution Climate Simulations and Quantifying Rare Events Statistics

A non-intrusive machine learning framework for debiasing long-time coarse resolution climate simulations and quantifying rare events statistics

Using deep learning to emulate and accelerate a radiative-transfer model

Towards Physically Consistent Deep Learning For Climate Model Parameterizations