Abstract:This paper develops new insights into quantitative methods for the validation of computational model prediction. Four types of methods are investigated, namely classical and Bayesian hypothesis testing, a reliability-based method, and an area metric-based method. Traditional Bayesian hypothesis testing is extended based on interval hypotheses on distribution parameters and equality hypotheses on probability distributions, in order to validate models with deterministic/stochastic output for given inputs. Formulations and implementation details are outlined for both equality and interval hypotheses. Two types of validation experiments are considered—fully characterized (all the model/experimental inputs are measured and reported as point values) and partially characterized (some of the model/experimental inputs are not measured or are reported as intervals). Bayesian hypothesis testing can minimize the risk in model selection by properly choosing the model acceptance threshold, and its results can be used in model averaging to avoid Type I /II errors. It is shown that Bayesian interval hypothesis testing, the reliability-based method, and the area metric-based method can account for the existence of directional bias, where the mean predictions of a numerical model may be consistently below or above the corresponding experimental observations. It is also found that under some specific conditions, the Bayes factor metric in Bayesian equality hypothesis testing and the reliability-based metric can both be mathematically related to the p-value metric in classical hypothesis testing. Numerical studies are conducted to apply the above validation methods to gas damping prediction for radio frequency (RF) micro-electro-mechanical-system (MEMS) switches. The model of interest is a general polynomial chaos (gPC) surrogate model constructed based on expensive runs of a physics-based simulation model, and validation data are collected from fully characterized experiments.

A Bayesian Inference Method for Model Extrapolation Together with Qualitative Knowledge

Evidential Model Validation under Epistemic Uncertainty

Structural Modelling and Bayesian Inference for Model Validation and Confidence Extrapolation

Towards Improving the Predictive Capability of Computer Simulations by Integrating Inverse Uncertainty Quantification and Quantitative Validation with Bayesian Hypothesis Testing

Surrogate-Based Inverse Modeling of the Hydrological System: An Adaptive Approach Considering Surrogate Structural Error.

Model Validation Via Uncertainty Propagation and Data Transformations

Quantitative model validation techniques: New insights

Bayesian methodology for reliability model acceptance

Predictive Capability Maturity Quantification Using Bayesian Network

Improving Data-Driven Inferential Sensor Modeling by Industrial Knowledge: A Bayesian Perspective

Knowledge-Based Method for the Validation of Complex Simulation Models

An Adaptive Sequential Experiment Design Method for Model Validation

Expert Knowledge-Driven Bayesian Network Modeling for Marine Disaster Assessment Under the Small Sample Condition

Bayesian probabilistic propagation of hybrid uncertainties: Estimation of response expectation function, its variable importance and bounds

A Bayesian approach for parameter estimation and prediction using a computationally intensive model

Incorporating Prior Knowledge for Quantifying and Reducing Model-Form Uncertainty in Rans Simulations

A New Fuzzy Bayesian Inference Approach for Risk Assessments

A Reliability Model Validation Method For Mitigating The Effects Of Measurement Uncertainty

Extrapolation-Aware Nonparametric Statistical Inference

Model-based Validation as Probabilistic Inference