An efficient sequential sampling approach based on cross-validation for deterministic computer simulations
Xiwen Cai,Haobo Qiu,Liang Gao
2015-01-01
Abstract:As is known, sampling strategies play a very important role in building accurate approximation models, which are widely used in complex system design to facilitate engineering optimization that involves computationally expensive simulations. Therefore to alleviate the computation burden, a new sequential sampling approach based on cross-validation for single response adaptive design of deterministic computer simulations is proposed in this paper. The proposed approach is called PMCV, for point to model cross validation. PMCV constructs an error model of mean squared error (MSE) based on cross-validation, which is made by the unobserved points plus current samples points. Here the unobserved points’ response can be predicted by the current model built by current samples and can be taken as the known sample points for model construction. It is considered that, for the unobserved points, their predicted response deviating from the true response lager will lead to larger model error. Then the new sample points can be calculated through maximizing MSE error model by a global optimizing algorithm like DIRECT. While only maximizing the error model may result in clustered samples in the design space, which is not desired in building an accurate model. Thus the space filling property of samples is considered to choose new points by constraining the minimum distance between any two different sample points. Compared with the two known sampling methods based on average cross-validation error model in Jin (2002) and an accumulative error model in Li (2010), the advantage of the proposed method is that it can adequately utilize the information of unobserved points, which may be important to improve sampling efficiency. In order to demonstrate the applicability of the proposed design of experiment (DOE) approach, numerous numerical and engineering examples are tested. And for all the testing examples, the radial basis function (RBF) is used as the metamodel for approximation. But the proposed approach seems to be adaptive for different metamodels. According to different metamodel accuracy performance metrics, results from these examples show that for the same number of simulation evaluations the proposed DOE method performs better for most of the test problems compared with three other DOE methods selected from literature. Besides, it is found, from the perspective of the ability to utilize model information in sequential sampling process, that PMCV method also has a great advantage among the compared DOE methods based on cross-validation. References [1] Jin, R., Chen,W. and SudjiantoA. (2002) On sequential sampling for global metamedling in engineering deign.ASME2002 design engineering technical conferences, 29 September–2 October,Montreal, Canada. DETC 2002/DAC-34092. [2] Li G, Aute V, Azarm S. (2010) An accumulative error based adaptive design of experiments for offline metamodeling[J]. Structural and Multidisciplinary Optimization, 40(1-6): 137-155.