Physiology-informed regularization enables training of universal differential equation systems for biological applications

Max de Rooij,Balazs Erdos,Natal van Riel,Shauna O'Donovan

DOI: https://doi.org/10.1101/2024.05.28.596164

2024-06-01

Abstract:Systems biology tackles the challenge of understanding the high complexity in the internal regulation of homeostasis in the human body through mathematical modelling. These models can aid in the discovery of disease mechanisms and potential drug targets. However, on one hand the development and validation of knowledge-based mechanistic models is time-consuming and does not scale well with increasing features in medical data. On the other hand, more data-driven approaches such as machine learning models require large volumes of data to produce generalizable models. The integration of neural networks and mechanistic models, forming universal differential equation (UDE) models, enables the automated learning of unknown model terms with less data than the neural network alone. Nevertheless, estimating parameters for these hybrid models remains difficult with sparse data and limited sampling durations that are common in biological applications. In this work, we propose the use of physiology-informed regularization, penalizing biologically implausible model behavior to guide the UDE towards more physiologically plausible regions of the solution space. In a simulation study we show that physiology-informed regularization not only results in a more accurate forecasting of model behaviour, but also supports training with less data. We also applied this technique to learn a representation of the rate of glucose appearance in the glucose minimal model using meal response data measured in healthy people. In that case, the inclusion of regularization reduces variability between UDE-embedded neural networks that were trained from different initial parameter guesses.

Systems Biology

What problem does this paper attempt to address?

This paper focuses on the issues encountered when training Universal Differential Equation (UDE) systems in biological applications. Traditional mechanistic model development is time-consuming and not suitable for high-dimensional medical data, while machine learning models require large amounts of data. UDE models combine neural networks and mechanistic models to learn unknown model terms with limited data. However, parameter estimation difficulties still exist, especially in cases of sparse biological data and limited sampling time.

To address these issues, the paper proposes a "physiological information regularization" approach, which guides the UDE model towards more physiologically plausible solution spaces by penalizing biologically unreasonable behaviors. The research shows that this approach not only improves the accuracy of model predictions but also reduces the amount of required data. In both simulated studies and applications on human data, such as learning the rate of postprandial blood glucose appearance, physiological information regularization demonstrates advantages by reducing variations between UDE embedded neural networks caused by different initial parameter guesses.

In summary, the paper aims to address how to improve the training of UDE models in situations where biological data is limited and sparse, through regularization techniques. This allows the models to better capture the dynamic behavior of biological systems and improve prediction accuracy and generalization capabilities.