Abstract:Approximation models (or surrogate models) provide an efficient substitute to expensive physical simulations and an efficient solution to the lack of physical models of system behavior. However, it is challenging to quantify the accuracy and reliability of such approximation models in a region of interest or the overall domain without additional system evaluations. Standard error measures, such as the mean squared error, the cross-validation error, and the Akaikes information criterion, provide limited (often inadequate) information regarding the accuracy of the final surrogate. This paper introduces a novel and model independent concept to quantify the level of errors in the function value estimated by the final surrogate in any given region of the design domain. This method is called the Regional Error Estimation of Surrogate (REES). Assuming the full set of available sample points to be fixed, intermediate surrogates are iteratively constructed over a sample set comprising all samples outside the region of interest and heuristic subsets of samples inside the region of interest (i.e., intermediate training points). The intermediate surrogate is tested over the remaining sample points inside the region of interest (i.e., intermediate test points). The fraction of sample points inside region of interest, which are used as intermediate training points, is fixed at each iteration, with the total number of iterations being pre-specified. The estimated median and maximum relative errors within the region of interest for the heuristic subsets at each iteration are used to fit a distribution of the median and maximum error, respectively. The estimated statistical mode of the median and the maximum error, and the absolute maximum error are then represented as functions of the density of intermediate training points, using regression models. The regression models are then used to predict the expected median and maximum regional errors when all the sample points are used as training points. Standard test functions and a wind farm power generation problem are used to illustrate the effectiveness and the utility of such a regional error quantification method.

Surrogate-Based Design Optimization with Adaptive Sequential Sampling

Quantifying Regional Error in Surrogates by Modeling Its Relationship with Sample Density

A New Sequential Sampling Method for Surrogate Modeling Based on a Hybrid Metric

General-Surrogate Adaptive Sampling Using Interquartile Range for Design Space Exploration

Sequential design for surrogate modeling in Bayesian inverse problems

Adaptive resampling strategy of sequential optimization based on radial basis function surrogate model

Characterizing Uncertainty Attributable to Surrogate Models

Surrogate-Based Inverse Modeling of the Hydrological System: An Adaptive Approach Considering Surrogate Structural Error.

A Sequential Sampling Approach for Multi-Fidelity Surrogate Modeling-Based Robust Design Optimization

Adaptive sequential sampling for surrogate model generation with artificial neural networks

A Novel Approach to Simultaneous Selection of Surrogate Models, Constitutive Kernels, and Hyper-parameter Values

Development of surrogate models in reliability-based design optimization: A review

Adaptive Gaussian Process Regression for Efficient Building of Surrogate Models in Inverse Problems

Surrogate-assisted Global Transfer Optimization Based on Adaptive Sampling Strategy

Conservative Surrogate Model Using Weighted Kriging Variance for Sampling-Based RBDO

A Novel Adaptive Region-Based Global Optimization Method for High Dimensional Problem

A Sequential Sampling Generation Method for Multi-Fidelity Model Based on Voronoi Region and Sample Density

Global Optimization Method Using Adaptive and Parallel Ensemble of Surrogates for Engineering Design Optimization

Reliability-based Design Optimization Using Adaptive Surrogate Model and Importance Sampling-Based Modified SORA Method.

Adaptive Sampling with Varying Sampling Cost for Design Space Exploration

A Reliability-Based Optimization Method Using Sequential Surrogate Model and Monte Carlo Simulation