Abstract:We explore how neural differential equations (NDEs) may be trained on highly resolved fluid-dynamical models of unresolved scales providing an ideal framework for data-driven parameterizations in climate models. NDEs overcome some of the limitations of traditional neural networks (NNs) in fluid dynamical applications in that they can readily incorporate conservation laws and boundary conditions and are stable when integrated over time. We advocate a method that employs a 'residual' approach, in which the NN is used to improve upon an existing parameterization through the representation of residual fluxes which are not captured by the base parameterization. This reduces the amount of training required and providing a method for capturing up-gradient and nonlocal fluxes. As an illustrative example, we consider the parameterization of free convection of the oceanic boundary layer triggered by buoyancy loss at the surface. We demonstrate that a simple parameterization of the process - convective adjustment - can be improved upon by training a NDE against highly resolved explicit models, to capture entrainment fluxes at the base of the well-mixed layer, fluxes that convective adjustment itself cannot represent. The augmented parameterization outperforms existing commonly used parameterizations such as the K-Profile Parameterization (KPP). We showcase that the NDE performs well independent of the time-stepper and that an online training approach using differentiable simulation via the Julia scientific machine learning software stack improves accuracy by an order-of-magnitude. We conclude that NDEs provide an exciting route forward to the development of representations of sub-grid-scale processes for climate science, opening up myriad new opportunities.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the parameterization of un - resolved physical processes (missing physics) in climate models, especially the vertical mixing processes related to free convection in the ocean boundary layer. Specifically, traditional parameterization methods have limitations in representing these small - scale physical processes, such as structural defects and uncertainties. Therefore, the author proposes a data - driven parameterization framework based on Neural Differential Equations (NDEs) to improve existing parameterization schemes. ### Main problems: 1. **Un - resolved small - scale physical processes**: Climate models cannot resolve every cloud or eddy in the ocean, but these small - scale processes have an important impact on the Earth's climate. 2. **Limitations of existing parameterizations**: Traditional parameterization methods rely on empirical selection, have uncertainties and structural defects, and are difficult to accurately capture complex physical phenomena, such as free convection and turbulent mixing in the ocean boundary layer. 3. **How to improve the accuracy of parameterization**: A new method is needed to more accurately represent these un - resolved processes, especially in terms of free convection and turbulent mixing. ### Solutions: The author proposes to use Neural Differential Equations (NDEs) to enhance existing parameterization schemes. The specific steps are as follows: - **Data - driven parameterization framework**: By combining existing parameterizations with neural networks and using high - resolution hydrodynamic simulation data for online training. - **Advantages of Neural Differential Equations**: NDEs can easily incorporate conservation laws and boundary conditions and remain stable during long - time integration, thus overcoming the limitations of traditional neural networks in hydrodynamic applications. - **Improve free convection parameterization**: Train NDEs to capture up - gradient and non - local fluxes that cannot be represented by existing parameterizations (such as convective adjustment), especially the entrainment flux at the bottom of the mixing layer. ### Formula presentation: - **Temperature tendency equation**: \[ \frac{\partial T}{\partial t}=\nabla \cdot F \] where \( F = F_{\text{param}}+F_{\text{missing}} \), \( F_{\text{param}} \) is the flux calculated by the existing parameterization, and \( F_{\text{missing}} \) is the missing flux predicted by the neural network. - **Thermal flux of free convection**: \[ w'T' = w'T'|_{\text{CA}}+w'T'|_{\text{missing}}=-\kappa_{\text{CA}}\frac{\partial T}{\partial z}+w'T'|_{\text{missing}} \] where \( w'T'|_{\text{CA}} \) is the turbulent vertical thermal flux predicted by convective adjustment, and \( w'T'|_{\text{missing}} \) is the missing part learned by the neural network. Through this method, the author demonstrates the potential of NDEs in improving the parameterization of free convection in the ocean boundary layer and proposes a general framework that can be used for the parameterization of other un - resolved processes.

Capturing missing physics in climate model parameterizations using neural differential equations

A Physics-Constrained Neural Differential Equation Framework for Data-Driven Snowpack Simulation

Parameterizing Vertical Mixing Coefficients in the Ocean Surface Boundary Layer using Neural Networks

An Analysis of Deep Learning Parameterizations for Ocean Subgrid Eddy Forcing

Towards Physically Consistent Deep Learning For Climate Model Parameterizations

Neural Ordinary Differential Equations for Data-Driven Reduced Order Modeling of Environmental Hydrodynamics

Use of neural networks for stable, accurate and physically consistent parameterization of subgrid atmospheric processes with good performance at reduced precision

Neural Infalling Cloud Equations (NICE): Increasing the Efficacy of Subgrid Models and Scientific Equation Discovery using Neural ODEs and Symbolic Regression

Physics-Informed Neural Networks for Multiphysics Data Assimilation with Application to Subsurface Transport

Neural Ideal Large Eddy Simulation: Modeling Turbulence with Neural Stochastic Differential Equations

Neural Network Parameterization of Subgrid‐Scale Physics From a Realistic Geography Global Storm‐Resolving Simulation

Using Deep Learning for Convection Parameterization

A non-intrusive machine learning framework for debiasing long-time coarse resolution climate simulations and quantifying rare events statistics

Simulating Atmospheric Processes in Earth System Models and Quantifying Uncertainties with Deep Learning Multi-Member and Stochastic Parameterizations

Interpreting and Stabilizing Machine-learning Parametrizations of Convection

A Non‐Intrusive Machine Learning Framework for Debiasing Long‐Time Coarse Resolution Climate Simulations and Quantifying Rare Events Statistics

Graph Neural Networks and Differential Equations: A hybrid approach for data assimilation of fluid flows

Data Imbalance, Uncertainty Quantification, and Generalization via Transfer Learning in Data-driven Parameterizations: Lessons from the Emulation of Gravity Wave Momentum Transport in WACCM

Combining data assimilation and machine learning to estimate parameters of a convective-scale model

A probabilistic framework for learning non-intrusive corrections to long-time climate simulations from short-time training data