Abstract:Abstract We are concerned with the efficiency of stochastic gradient estimation methods for large-scale nonlinear optimization in the presence of uncertainty. These methods aim to estimate an approximate gradient from a limited number of random input vector samples and corresponding objective function values. Ensemble methods usually employ Gaussian sampling to generate the input samples. It is known from the optimal design theory that the quality of sample-based approximations is affected by the distribution of the samples. We therefore evaluate six different sampling strategies to optimization of a high-dimensional analytical benchmark optimization problem, and, in a second example, to optimization of oil reservoir management strategies with and without geological uncertainty. The effectiveness of the sampling strategies is analyzed based on the quality of the estimated gradient, the final objective function value, the rate of the convergence, and the robustness of the gradient estimate. Based on the results, an improved version of the stochastic simplex approximate gradient method is proposed based on U E ( s 2 ) sampling designs for supersaturated cases that outperforms all alternative approaches. We additionally introduce two new strategies that outperform the U E ( s 2 ) designs previously suggested in the literature.

Optimal Importance Sampling for Simulation of Lévy Processes

Efficient Importance Sampling for Rare Event Simulation with Applications

Sample Average Approximation Method for a Class of Stochastic Variational Inequality Problems.

Importance sampling for jump processes and applications to finance

Improving Importance Sampling Method in Structural Reliability

Nonasymptotic Bounds for Suboptimal Importance Sampling

Efficient Rare-Event Simulation for Multiple Jump Events in Regularly Varying Lévy Processes with Infinite Activities

Randomly Shifted Lattice Rules with Importance Sampling and Applications

Importance Sampled Stochastic Optimization for Variational Inference

An Adaptive Importance Sampling for Locally Stable Point Processes

Variational Learning of Gaussian Process Latent Variable Models through Stochastic Gradient Annealed Importance Sampling

A Novel Adaptive Importance Sampling Algorithm for Bayesian Model Updating

Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problem

Efficient large deviation estimation based on importance sampling

Improved sampling strategies for ensemble-based optimization

Iterative Importance Sampling Algorithms for Parameter Estimation

On importance sampling and independent Metropolis-Hastings with an unbounded weight function

A Unified Particle-Optimization Framework for Scalable Bayesian Sampling

Achieving Efficiency in Black Box Simulation of Distribution Tails with Self-structuring Importance Samplers

A General Framework for Importance Sampling with Latent Markov Processes

Subsampled Optimization: Statistical Guarantees, Mean Squared Error Approximation, and Sampling Method