Abstract:Universal differential equations (UDEs) leverage the respective advantages of mechanistic models and artificial neural networks and combine them into one dynamic model. However, these hybrid models can suffer from unrealistic solutions, such as negative values for biochemical quantities. We present non-negative UDE (nUDEs), a constrained UDE variant that guarantees non-negative values. Furthermore, we explore regularisation techniques to improve generalisation and interpretability of UDEs.

What problem does this paper attempt to address?

The paper primarily aims to address the issues encountered when using Universal Differential Equations (UDEs) to model biological systems, particularly how to ensure that the model predictions are biologically meaningful and how to improve the generality and interpretability of UDEs. Specifically, the paper addresses the following key issues: 1. **Non-negativity Guarantee**: In biological system modeling, certain quantities such as molecular concentrations or population sizes must be non-negative. However, traditional UDEs may produce unreasonable negative solutions. Therefore, the authors propose Non-negative UDEs (nUDEs), which use constraint mechanisms to ensure that these quantities always remain non-negative. 2. **Overfitting Problem**: UDEs, as a method that combines mechanistic models and machine learning, perform well when dealing with incomplete information. However, this hybrid model is prone to overfitting, where the model excessively adapts to noise or anomalies in the training data, thereby harming its generalization ability. The paper introduces two regularization techniques—parameter regularization and output regularization—to improve the generalization ability and interpretability of UDEs. 3. **Experimental Validation**: The paper validates the effectiveness of the proposed nUDEs and regularization methods through examples of synthetic and actual biological systems. For instance, in the Lotka-Volterra model, it demonstrates that nUDEs can effectively avoid negative solutions; while in the Boehm model, it verifies the importance of regularization in improving model quality and reducing overfitting. In summary, this study aims to improve the application of UDEs in biological system modeling by introducing nUDEs and regularization strategies, ensuring the biological reasonableness of model predictions, and enhancing the model's generalization ability.

Non-Negative Universal Differential Equations With Applications in Systems Biology

Universal differential equations for systems biology: Current state and open problems

Universal Differential Equations as a Common Modeling Language for Neuroscience

Assessment of Uncertainty Quantification in Universal Differential Equations

Optimal Experimental Design for Universal Differential Equations

An analysis of Universal Differential Equations for data-driven discovery of Ordinary Differential Equations

Physiology-informed regularization enables training of universal differential equation systems for biological applications

On Neural Differential Equations

Neural Differential Algebraic Equations

Hybrid Neural Differential Equations to Model Unknown Mechanisms and States in Biology

Uncertainty Quantification in Neural Differential Equations

Identifying Latent Dynamic Components in Biological Systems

Stabilized Neural Differential Equations for Learning Dynamics with Explicit Constraints

Data2Dynamics: a Modeling Environment Tailored to Parameter Estimation in Dynamical Systems.

Bayesian Neural Ordinary Differential Equations

Non-negative distributed regression for data inference in wireless sensor networks

A Survey on Solving and Discovering Differential Equations Using Deep Neural Networks

Learning COVID-19 Regional Transmission Using Universal Differential Equations in a SIR model

Identifiability of Differential-Algebraic Systems

Emulating Recombination with Neural Networks using Universal Differential Equations

Solution of Infinite Systems of Differential Equations of Possible Interest in Evolutionary Theories