Abstract:This paper develops new insights into quantitative methods for the validation of computational model prediction. Four types of methods are investigated, namely classical and Bayesian hypothesis testing, a reliability-based method, and an area metric-based method. Traditional Bayesian hypothesis testing is extended based on interval hypotheses on distribution parameters and equality hypotheses on probability distributions, in order to validate models with deterministic/stochastic output for given inputs. Formulations and implementation details are outlined for both equality and interval hypotheses. Two types of validation experiments are considered—fully characterized (all the model/experimental inputs are measured and reported as point values) and partially characterized (some of the model/experimental inputs are not measured or are reported as intervals). Bayesian hypothesis testing can minimize the risk in model selection by properly choosing the model acceptance threshold, and its results can be used in model averaging to avoid Type I /II errors. It is shown that Bayesian interval hypothesis testing, the reliability-based method, and the area metric-based method can account for the existence of directional bias, where the mean predictions of a numerical model may be consistently below or above the corresponding experimental observations. It is also found that under some specific conditions, the Bayes factor metric in Bayesian equality hypothesis testing and the reliability-based metric can both be mathematically related to the p-value metric in classical hypothesis testing. Numerical studies are conducted to apply the above validation methods to gas damping prediction for radio frequency (RF) micro-electro-mechanical-system (MEMS) switches. The model of interest is a general polynomial chaos (gPC) surrogate model constructed based on expensive runs of a physics-based simulation model, and validation data are collected from fully characterized experiments.

An Evidence Theory-Based Validation Method for Models with Multivariate Outputs and Uncertainty

Model Validation Via Uncertainty Propagation and Data Transformations

A New Validation Metric for Models with Correlated Responses Using Limited Experimental and Simulation Data

A Modified Model of Failure Mode and Effects Analysis Based on Generalized Evidence Theory

An Improved Belief Entropy–based Uncertainty Management Approach for Sensor Data Fusion

A Contribution to Validating of Simulation Models in Production and Logistics

The Validation of A Microscopic Simulation Model: A Methodological Case Study

Quantitative model validation techniques: New insights

A New Methodology For Simulation Validation And Model Modification

Evidential FMEA method for human reliability assessment

Failure Mode and Effects Analysis on the Air System of an Aero Turbofan Engine Using the Gaussian Model and Evidence Theory

Potential of statistical model verification, validation and uncertainty quantification in automotive vehicle dynamics simulations: a review

Incorporating uncertainty into the validation of automotive crash simulation models for component analysis

An Improved Failure Mode and Effects Analysis Method Using Belief Jensen–Shannon Divergence and Entropy Measure in the Evidence Theory

Modeling of CNC Machine Tool Thermal Errors Based on LS-SVM within Bayesian Evidence Framework

An empirical learning-based validation procedure for simulation workflow

Optimal Design of Validation Experiments for the Prediction of Quantities of Interest

Knowledge-based model validation using a custom metric

Efficient Validation of Self-Adaptive Applications by Counterexample Probability Maximization.

Fusion of expert uncertain assessment in FMEA based on the negation of basic probability assignment and evidence distance