Abstract:Sequential sampling methods have gained significant attention due to their ability to iteratively construct surrogate models by sequentially inserting new samples based on existing ones. However, efficiently and accurately creating surrogate models for high-dimensional, nonlinear, and multimodal problems is still a challenging task. This paper proposes a new sequential sampling method for surrogate modeling based on a hybrid metric, specifically making the following three contributions: (1) a hybrid metric is developed by integrating the leave-one-out cross-validation error, the local nonlinearity, and the relative size of Voronoi regions using the entropy weights, which well considers both the global exploration and local exploitation of existing samples; (2) a Pareto-TOPSIS strategy is proposed to first filter out unnecessary regions and then efficiently identify the sensitive region within the remaining regions, thereby improving the efficiency of sensitive region identification; and (3) a prediction-error-and-variance (PE&V) learning function is proposed based on the prediction error and variance of the intermediate surrogate models to identify the new sample to be inserted in the sensitive region, ultimately improving the efficiency of the sequential sampling process and the accuracy of the final surrogate model. The proposed sequential sampling method is compared with four state-of-the-art sequential sampling methods for creating Kriging surrogate models in seven numerical cases and one real-world engineering case of a cutterhead of a tunnel boring machine. The results show that compared with the other four methods, the proposed sequential sampling method can more quickly and robustly create an accurate surrogate model using a smaller number of samples.

Accelerating Stratified Sampling SGD by Reconstructing Strata

Adaptive Client Sampling in Federated Learning via Online Learning with Bandit Feedback

Lsh-sampling Breaks the Computation Chicken-and-egg Loop in Adaptive Stochastic Gradient Estimation

Asynchronous Accelerated Stochastic Gradient Descent.

A New Sequential Sampling Method for Surrogate Modeling Based on a Hybrid Metric

Stochastic gradient descent, weighted sampling, and the randomized Kaczmarz algorithm

ADASS: Adaptive Sample Selection for Training Acceleration

Accelerating Stochastic Gradient Descent Using Antithetic Sampling.

Optimal Adaptive and Accelerated Stochastic Gradient Descent

One Backward from Ten Forward, Subsampling for Large-Scale Deep Learning

Adaptive Variance Reducing for Stochastic Gradient Descent.

Enhancing Stochastic Gradient Descent: A Unified Framework and Novel Acceleration Methods for Faster Convergence

Accelerating Minibatch Stochastic Gradient Descent Using Typicality Sampling

Stratification and Optimal Resampling for Sequential Monte Carlo

Aiming towards the minimizers: fast convergence of SGD for overparametrized problems

IS-ASGD

Towards Better Generalization of Deep Neural Networks via Non-Typicality Sampling Scheme

STSyn: Speeding Up Local SGD With Straggler-Tolerant Synchronization

Effective Gradient Sample Size via Variation Estimation for Accelerating Sharpness aware Minimization

Accelerated Doubly Stochastic Gradient Algorithm for Large-scale Empirical Risk Minimization