Abstract:Approximation models (or surrogate models) provide an efficient substitute to expensive physical simulations and an efficient solution to the lack of physical models of system behavior. However, it is challenging to quantify the accuracy and reliability of such approximation models in a region of interest or the overall domain without additional system evaluations. Standard error measures, such as the mean squared error, the cross-validation error, and the Akaikes information criterion, provide limited (often inadequate) information regarding the accuracy of the final surrogate. This paper introduces a novel and model independent concept to quantify the level of errors in the function value estimated by the final surrogate in any given region of the design domain. This method is called the Regional Error Estimation of Surrogate (REES). Assuming the full set of available sample points to be fixed, intermediate surrogates are iteratively constructed over a sample set comprising all samples outside the region of interest and heuristic subsets of samples inside the region of interest (i.e., intermediate training points). The intermediate surrogate is tested over the remaining sample points inside the region of interest (i.e., intermediate test points). The fraction of sample points inside region of interest, which are used as intermediate training points, is fixed at each iteration, with the total number of iterations being pre-specified. The estimated median and maximum relative errors within the region of interest for the heuristic subsets at each iteration are used to fit a distribution of the median and maximum error, respectively. The estimated statistical mode of the median and the maximum error, and the absolute maximum error are then represented as functions of the density of intermediate training points, using regression models. The regression models are then used to predict the expected median and maximum regional errors when all the sample points are used as training points. Standard test functions and a wind farm power generation problem are used to illustrate the effectiveness and the utility of such a regional error quantification method.

Quantifying Regional Error in Surrogates by Modeling Its Relationship with Sample Density

Error modeling for surrogates of dynamical systems using machine learning

Surrogate-Based Inverse Modeling of the Hydrological System: An Adaptive Approach Considering Surrogate Structural Error.

Evaluation of Failure Probability Via Surrogate Models

Surrogate Model Uncertainty Quantification for Active Learning Reliability Analysis

A New Sequential Sampling Method for Surrogate Modeling Based on a Hybrid Metric

Ensemble learning of multi-kernel Kriging surrogate models using regional discrepancy and space-filling criteria-based hybrid sampling method

General-Surrogate Adaptive Sampling Using Interquartile Range for Design Space Exploration

Comparative Studies of Error Metrics in Variable Fidelity Model Uncertainty Quantification

INVESTIGATION OF SURROGATE MODEL FOR UNCERTAINTY QUANTIFICATION OF NUCLEAR SYSTEM

A Novel Method for Ensemble of Surrogates Based on Global and Local Measures

Verification Methods for Surrogate Models

The Ensemble of Surrogate Model Based on Local and Global Errors

Research on Surrogate Model Based on Local Radial Point Interpolation Method

A Region-Segmentation Combining Surrogate Model Based on L-indicator and N-fold Cross-Validation Technique

Small Area Quantile Estimation

A General Failure-Pursuing Sampling Framework for Surrogate-Based Reliability Analysis

Surrogate Modeling for Spatially Distributed Fuel Cell Models With Applications to Uncertainty Quantification

Surrogate modeling for probability distribution estimation:uniform or adaptive design?

A Sequential Sampling Generation Method for Multi-Fidelity Model Based on Voronoi Region and Sample Density

Revised Regional Importance Measures in the Presence of Epistemic and Aleatory Uncertainties