Abstract:Our objective is to devise an efficient optimization algorithm for estimating optimal well controls during the production optimization step of closed-loop reservoir management where the optimization algorithm does not require an adjoint method for the calculation of derivatives. Research within the TUPREP group strongly suggests that gradient-based algorithms will be the most efficient and reliable methods for estimating well controls to maximize the net present value (NPV), or cumulative oil recovery of an oil field. However, most major oil companies use commercial reservoir simulators to make future predictions of production, and such commercial simulators generally lack efficient gradient-based production optimization algorithms where computational efficiency demands that the gradient of the cost function (e.g., NPV or cumulative oil production over the life of the field) be computed by an adjoint method. Adding an adjoint solution without access to the source code is not feasible. Therefore, it is important to implement optimization algorithms that can be easily coupled with a commercial simulator simply by using the simulator as a black box. While many such gradient-free optimization algorithms are available, based on the excellent performance of gradient-based methods, it is desirable to develop a procedure that mimics the property of methods that use a gradient. One such method is the simultaneous perturbation stochastic approximation (SPSA) method which estimates a stochastic gradient by a procedure that is far more efficient than estimating the derivative with a finite difference approximation. The purpose of this study is to develop a computationally efficient version of the SPSA algorithm for the constrained production optimization problem involved in the optimal control step of closed-loop reservoir management. The SPSA is a stochastic optimization algorithm, which obtains its stochastic gradient by simultaneously perturbing all components of the well control vector in a stochastic way. Although the SPSA gradient is stochastic, it is always uphill for a sufficiently small perturbation size and its expectation is equal to the true gradient as the perturbation size goes to zero. The most challenging problem in production optimization is to honor state-control constraints which are nonlinear. The SPSA algorithm has been applied to production optimization problems with only bound constraints and implementations used in previous published work were computationally inefficient. In this study, the focus will be developing and implementing a computationally efficient SPSA algorithm for production optimization problems with general equality, inequality and bound constraints. Here, the bound constraints are converted into the new unbounded control variables using a log-transformation. Although the results are not shown in this work, simply truncating control variables at their bounds consistently yields slightly lower NPVs values than are obtained with the log-transform method used here. All constraints are incorporated into the objective function to be maximized using the augmented Lagrangian function. The advantage of this method is that the general inequality and equality constraints can conceptually be handled using suitable values of the penalty parameter and Lagrange multipliers. In addition to the basic SPSA algorithm, which uses samples from the symmetric Bernoulli distribution as perturbations, a SPSA type algorithm that uses samples from Gaussian distribution is also tested. To improve the efficiency of the SPSA algorithm, a combined search direction SPSA algorithm is proposed, in which the search direction is a combination of the current SPSA gradient and the "best" SPSA gradient or some "good" former SPSA gradients calculated in the last few iterations.

Parallel Implementation of Simultaneous Perturbation Stochastic Approximation with Adaptive Step Sizes.

Parallel adaptive hybrid genetic optimization algorithm and its application

Improved Particle Swarm Optimizer for Problems with Variable Evaluation Time: Application to Asymptotic Homogenization

Parallel Stochastic Optimization Framework for Large-Scale Non-Convex Stochastic Problems

A Method of Parameter Optimization for Particle Swarm Optimization Based on Stochastic Processes

Online Successive Convex Approximation for Two-Stage Stochastic Nonconvex Optimization

Parallel Simultaneous Perturbation Optimization

Optimization with Discrete Simultaneous Perturbation Stochastic Approximation Using Noisy Loss Function Measurements

Application of spsa-type algorithms to production optimization

Stochastic Successive Convex Approximation for Non-Convex Constrained Stochastic Optimization

Hierarchical Parallel Simulated Annealing and Its Applications

Parallel asynchronous particle swarm optimization

DSPG: Decentralized Simultaneous Perturbations Gradient Descent Scheme

Comparison study of the combination of the SPSA algorithm and the PSO algorithm

Stochastic Primal Dual Hybrid Gradient Algorithm with Adaptive Step-Sizes

Stochastic Primal–Dual Hybrid Gradient Algorithm with Adaptive Step Sizes

BiSLS/SPS: Auto-tune Step Sizes for Stable Bi-level Optimization

Guided-SPSA: Simultaneous Perturbation Stochastic Approximation assisted by the Parameter Shift Rule

A Parallel Stochastic Approximation Method for Nonconvex Multi-Agent Optimization Problems

Adaptive smoothing mini-batch stochastic accelerated gradient method for nonsmooth convex stochastic composite optimization

A modified particle swarm optimization using adaptive strategy