Abstract:All fields of science depend on mathematical models. Occam's razor refers to the principle that good models should exclude parameters beyond those minimally required to describe the systems they represent. This is because redundancy can lead to incorrect estimates of model parameters from data, and thus inaccurate or ambiguous conclusions. Here, we show how deep learning can be powerfully leveraged to apply Occam's razor to model parameters. Our method, FixFit, uses a feedforward deep neural network with a bottleneck layer to characterize and predict the behavior of a given model from its input parameters. FixFit has three major benefits. First, it provides a metric to quantify the original model's degree of complexity. Second, it allows for the unique fitting of data. Third, it provides an unbiased way to discriminate between experimental hypotheses that add value versus those that do not. In three use cases, we demonstrate the broad applicability of this method across scientific domains. To validate the method using a known system, we apply FixFit to recover known composite parameters for the Kepler orbit model and a dynamic model of blood glucose regulation. In the latter, we demonstrate the ability to fit the latent parameters to real data. To illustrate how the method can be applied to less well-established fields, we use it to identify parameters for a multi-scale brain model and reduce the search space for viable candidate mechanisms. Mathematical modeling is a pillar of scientific inquiry, bridging the gap between theory and experimental observations. However, in complex systems such as those pervasive in biology (e.g., gene regulatory networks, multi-scale brain interactions, and drug pharmacokinetics), different mechanisms can yield equally plausible explanations for the data. This ambiguity is not due to data limitations but rather to the equations that govern these systems. Both the interpretation and estimation of the parameters of these models are hindered by these intrinsic degeneracies. For this reason, we present a general tool that harnesses the power of deep learning to automatically identify and rectify these ambiguities, allowing a broader range of models to be precisely determined by experimental data. We show, for example, that our method provides novel insights into the features of multi-scale brain dynamics that can be learned from functional neuroimaging. This represents a crucial initial step toward characterizing the mechanistic insights that different types of experiments can provide.

Bayesian Occam's Razor to Optimize Models for Complex Systems

Identification of Optimal Strategies for State Transition of Complex Biological Networks.

Identifying Bayesian Optimal Experiments for Uncertain Biochemical Pathway Models

Generalized Parameter Estimation Method for Model-Based Real Time Optimization

Achieving Occam's razor: Deep learning for optimal model reduction

Modeling and Optimization in Complex Systems Based on Computational Intelligence.

Enhanced Bayesian Optimization via Preferential Modeling of Abstract Properties

Mathematical basis and toolchain for hierarchical optimization of biochemical networks

Knowledge-based modeling of simulation behavior for Bayesian optimization

Bayesian uncertainty analysis for complex systems biology models: emulation, global parameter searches and evaluation of gene functions

POPE: post optimization posterior evaluation of likelihood free models

Estimating cellular parameters through optimization procedures: elementary principles and applications

Bayesian prediction for physical models with application to the optimization of the synthesis of pharmaceutical products using chemical kinetics

OptEx: An integrated framework for experimental design and combustion kinetic model optimization

Efficient construction of Bayes optimal designs for stochastic process models

BayesOpt: A Library for Bayesian optimization with Robotics Applications

Real-time optimization meets Bayesian optimization and derivative-free optimization: A tale of modifier adaptation

Global Optimization Approach for Parameter Estimation in Stochastic Dynamic Models of Biosystems

Bayesian Optimization with Additive Kernels for a Stepwise Calibration of Simulation Models for Cost-Effectiveness Analysis

Risk-averse optimization of genetic circuits under uncertainty

A parsimonious model of blood glucose homeostasis