Abstract:This paper develops new insights into quantitative methods for the validation of computational model prediction. Four types of methods are investigated, namely classical and Bayesian hypothesis testing, a reliability-based method, and an area metric-based method. Traditional Bayesian hypothesis testing is extended based on interval hypotheses on distribution parameters and equality hypotheses on probability distributions, in order to validate models with deterministic/stochastic output for given inputs. Formulations and implementation details are outlined for both equality and interval hypotheses. Two types of validation experiments are considered—fully characterized (all the model/experimental inputs are measured and reported as point values) and partially characterized (some of the model/experimental inputs are not measured or are reported as intervals). Bayesian hypothesis testing can minimize the risk in model selection by properly choosing the model acceptance threshold, and its results can be used in model averaging to avoid Type I /II errors. It is shown that Bayesian interval hypothesis testing, the reliability-based method, and the area metric-based method can account for the existence of directional bias, where the mean predictions of a numerical model may be consistently below or above the corresponding experimental observations. It is also found that under some specific conditions, the Bayes factor metric in Bayesian equality hypothesis testing and the reliability-based metric can both be mathematically related to the p-value metric in classical hypothesis testing. Numerical studies are conducted to apply the above validation methods to gas damping prediction for radio frequency (RF) micro-electro-mechanical-system (MEMS) switches. The model of interest is a general polynomial chaos (gPC) surrogate model constructed based on expensive runs of a physics-based simulation model, and validation data are collected from fully characterized experiments.

A Quantitative Analysis of Electromagnetic Simulation Model Credibility

A Quantitative Credibility Evaluation Method of Dynamic Equivalent Model Based on Power Sensitivity

Credibility Evaluation of Uncertainty Analysis Results of Emc Simulation

The Application of the NIPC Method and the FSV Method on the Validation of EM Simulation under HIRF Environment

Research on Credibility Quantification of Complex Simulation Systems

Validity Evaluation Of The Uncertain Emc Simulation Results

Simulation Analysis and Belief Reliability Evaluation of Electromagnetic Performance of Electronic Products

A credibility evaluation method for complex simulation systems based on interactive network analysis

An Effective Evaluation of Credibility of Air Data Computer Simulation System

Methodology of Simulation Credibility Evluation for Complex Product Based on DS Evidence Theory

Akind of Mission Computer Simulation System Credibility Research Method

Application of Spectrum Analysis in the Research of Simulation Model Credibility

Electromagnetic Coupling Estimation Based on Hybrid Equivalent Dipoles and Field-Circuit Models

Quantum Mechanical Approach to Modelling Reliability of Sensor Reports

A Novel Integrated Approach for Simulation of Electromagnetic Susceptibility Problem

Quantum Mechanical Approach to Modeling Reliability of Sensor Reports

Study on the Credibility Quantification of Large Complex Simulation Systems

A Pattern-Based Validation Method for the Credibility Evaluation of Simulation Models.

An Evidence Theory-Based Validation Method for Models with Multivariate Outputs and Uncertainty

A Novel Credibility Quantification Method for Welch's Periodogram Analysis Result in Model Validation

Quantitative model validation techniques: New insights