Abstract:This paper develops new insights into quantitative methods for the validation of computational model prediction. Four types of methods are investigated, namely classical and Bayesian hypothesis testing, a reliability-based method, and an area metric-based method. Traditional Bayesian hypothesis testing is extended based on interval hypotheses on distribution parameters and equality hypotheses on probability distributions, in order to validate models with deterministic/stochastic output for given inputs. Formulations and implementation details are outlined for both equality and interval hypotheses. Two types of validation experiments are considered—fully characterized (all the model/experimental inputs are measured and reported as point values) and partially characterized (some of the model/experimental inputs are not measured or are reported as intervals). Bayesian hypothesis testing can minimize the risk in model selection by properly choosing the model acceptance threshold, and its results can be used in model averaging to avoid Type I /II errors. It is shown that Bayesian interval hypothesis testing, the reliability-based method, and the area metric-based method can account for the existence of directional bias, where the mean predictions of a numerical model may be consistently below or above the corresponding experimental observations. It is also found that under some specific conditions, the Bayes factor metric in Bayesian equality hypothesis testing and the reliability-based metric can both be mathematically related to the p-value metric in classical hypothesis testing. Numerical studies are conducted to apply the above validation methods to gas damping prediction for radio frequency (RF) micro-electro-mechanical-system (MEMS) switches. The model of interest is a general polynomial chaos (gPC) surrogate model constructed based on expensive runs of a physics-based simulation model, and validation data are collected from fully characterized experiments.

Bayesian methodology for reliability model acceptance

Combining Model Checking and Testing with an Application to Reliability Prediction and Distribution

A Bayesian nonparametric system reliability model which integrates multiple sources of lifetime information

A Bayesian model averaging method for software reliability modeling and assessment

A Bayesian Measure of Model Accuracy

Bayesian Inference of System Reliability for Multicomponent Stress-Strength Model under Marshall-Olkin Weibull Distribution

Model Validation Via Uncertainty Propagation and Data Transformations

Quantitative model validation techniques: New insights

Bayesian Safety Validation for Failure Probability Estimation of Black-Box Systems

Structural reliability analysis: A Bayesian perspective

Application of Bayesian Methods for Age-dependent Reliability Analysis

REAL-TIME RELIABILITY PREDICTION METHOD BASED ON MARKOV CHAIN MONTE CARLO

Bayesian approach to hazard rate models for early detection of warranty and reliability problems using upstream supply chain information

The Unnecessity of Assuming Statistically Independent Tests in Bayesian Software Reliability Assessments

A novel multi-stage precision reliability assessment method for mechanical system by Bayesian fusion

Reliability assessment for failure‐dependent and uncertain systems: A Bayesian network based on copula method and probability‐box

Robust Bayesian Reliability for Complex Systems under Prior-Data Conflict

Medical and Engineering Applications for Estimation and Prediction of a New Competing Risks Model: A Bayesian Approach

A Model for Censored Reliability Data with Two Dependent Failure Modes and Prediction of Future Failures