BayCANN: Streamlining Bayesian Calibration with Artificial Neural Network Metamodeling

Hawre Jalal,Fernando Alarid-Escudero

DOI: https://doi.org/10.48550/arXiv.2010.13452

2020-10-26

Abstract:Purpose: Bayesian calibration is theoretically superior to standard direct-search algorithm because it can reveal the full joint posterior distribution of the calibrated parameters. However, to date, Bayesian calibration has not been used often in health decision sciences due to practical and computational burdens. In this paper we propose to use artificial neural networks (ANN) as one solution to these limitations. Methods: Bayesian Calibration using Artificial Neural Networks (BayCANN) involves (1) training an ANN metamodel on a sample of model inputs and outputs, and (2) then calibrating the trained ANN metamodel instead of the full model in a probabilistic programming language to obtain the posterior joint distribution of the calibrated parameters. We demonstrate BayCANN by calibrating a natural history model of colorectal cancer to adenoma prevalence and cancer incidence data. In addition, we compare the efficiency and accuracy of BayCANN against performing a Bayesian calibration directly on the simulation model using an incremental mixture importance sampling (IMIS) algorithm. Results: BayCANN was generally more accurate than IMIS in recovering the "true" parameter values. The ratio of the absolute ANN deviation from the truth compared to IMIS for eight out of the nine calibrated parameters were less than one indicating that BayCANN was more accurate than IMIS. In addition, BayCANN took about 15 minutes total compared to the IMIS method which took 80 minutes. Conclusions: In our case study, BayCANN was more accurate than IMIS and was five-folds faster. Because BayCANN does not depend on the structure of the simulation model, it can be adapted to models of various levels of complexity with minor changes to its structure. We provide BayCANN's open-source implementation in R.

Methodology,Computation,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that in health decision - making science, although the Bayesian calibration method is theoretically superior to the standard direct search algorithm because it can reveal the joint posterior distribution of calibration parameters, this technique has not been widely used due to practical and computational burdens. Specifically, Bayesian calibration usually requires thousands of simulation runs and often needs to write models using probabilistic programming languages such as Stan or Bayesian inference Using Gibbs Sampling (BUGS), which is very technically challenging and also computationally expensive. To solve these problems, the author proposes a method of using artificial neural networks (ANN) as meta - models to simplify Bayesian calibration, called BayCANN. Through this method, an ANN meta - model can be trained to approximate the input - output relationship of the original model, and then this trained ANN meta - model is calibrated in a probabilistic programming language instead of directly calibrating the complete model to obtain the joint posterior distribution of calibration parameters. This method not only improves efficiency but also maintains the accuracy of Bayesian calibration. The paper demonstrates the effectiveness and efficiency of BayCANN by applying it to the calibration of a colorectal cancer natural history model and comparing the results with those of Bayesian calibration using the Incremental Mixture Importance Sampling (IMIS) algorithm directly on the simulation model. The results show that BayCANN is more accurate than IMIS in recovering the "true" parameter values and also significantly reduces the processing time. Moreover, since BayCANN does not depend on the specific structure of the simulation model, it can adapt to models of different complexities with only slight adjustments to its structure. The paper also provides the open - source implementation code of BayCANN, which is convenient for researchers to use in their own model calibration work.

BayCANN: Streamlining Bayesian Calibration with Artificial Neural Network Metamodeling

Emulator-Based Bayesian Calibration of the CISNET Colorectal Cancer Models

Bayesian calibration of stochastic agent based model via random forest

A Bayesian framework for functional calibration of expensive computational models through non-isometric matching

Achieving Well-Informed Decision-Making in Drug Discovery: A Comprehensive Calibration Study using Neural Network-Based Structure-Activity Models

Improving Neural Additive Models with Bayesian Principles

A Bayesian Neural Network-Based Method to Calibrate Microscopic Traffic Simulators

Calibration of stochastic, agent-based neuron growth models with Approximate Bayesian Computation

Bayesian Optimization with Additive Kernels for a Stepwise Calibration of Simulation Models for Cost-Effectiveness Analysis

Calibration-Aware Bayesian Learning

Bayesian Adaptive Calibration and Optimal Design

Calibrating Bayesian Learning via Regularization, Confidence Minimization, and Selective Inference

A neural network-based framework for financial model calibration

Model-Free Local Recalibration of Neural Networks

Coupling artificial neural networks with the artificial bee colony algorithm for global calibration of hydrological models

Recipes for calibration and validation of agent-based models in cancer biomedicine

Neural parameter calibration for large-scale multiagent models

A Bayesian approach for fitting semi-Markov mixture models of cancer latency to individual-level data

Bayesian Projected Calibration of Computer Models