Abstract:Efficient optimization methods coupling a stochastic evolutionary algorithm with a gradient based deterministic method are presented in this paper. Two kinds of hybridization are compared: one is a stochastic/deterministic alternate algorithm, the other is a stochastic/deterministic embedded algorithm. In the alternating algorithm, stochastic and deterministic optimizers are performed alternately as follows: some individuals are selected from the previous population, and sent to the deterministic algorithms for further optimization, then the improved individuals are inserted into the above population to form a new one for the stochastic algorithm. In the embedded hybridized algorithm, stochastic and deterministic optimization software are run in parallel and independently, the coupling between them is that the deterministic optimizer operates on a randomly selected individual (or the best individual) from the non evaluated population of the stochastic algorithm, then its outcome (new individual) is re-injected into the evaluated population. Moreover, a multilevel approximation (e.g. variable fidelity modeling, analysis and hierarchical approximated parameterization) is introduced in the algorithm, via a low fidelity modeling and rough parameterization to perform a search on large population at lower level, and a high fidelity modeling with detailed parameterization used at higher level. After a theoretical validation of the methods on mathematical test cases, the hybridized methods are successfully applied to the aerodynamic shape optimization of a fore-body of an hypersonic air breathing vehicle, providing both a significant acceleration in terms of parallel HPC performance and improved quality of the design.

Hybrid Modified Accelerated Gradient Method for Optimization Processes

A Modified Dai–Liao Conjugate Gradient Method Based on a Scalar Matrix Approximation of Hessian and Its Application

A HYBRID CONJUGATE GRADIENT METHOD FOR UNCONSTRAINED OPTIMIZATION WITH APPLICATION

Accelerated gradient methods for nonconvex nonlinear and stochastic programming

Hybrid Acceleration Scheme for Variance Reduced Stochastic Optimization Algorithms

Deterministic and Stochastic Accelerated Gradient Method for Convex Semi-Infinite Optimization

Accelerated nonlinear primal-dual hybrid gradient methods with applications to supervised machine learning

Fast Multiobjective Gradient Methods with Nesterov Acceleration via Inertial Gradient-Like Systems

Accelerated Bregman Proximal Gradient Methods for Relatively Smooth Convex Optimization

Multi-level Hybridized Optimization Methods Coupling Local Search Deterministic and Global Search Evolutionary Algorithms

Accelerated Gradient Methods with Gradient Restart: Global Linear Convergence

Accelerated gradient methods for nonconvex optimization: Escape trajectories from strict saddle points and convergence to local minima

Comparative Analysis of Gradient-Based Optimization Techniques Using Multidimensional Surface 3D Visualizations and Initial Point Sensitivity

Efficiency of Stochastic Coordinate Proximal Gradient Methods on Nonseparable Composite Optimization

Fast hybrid method for solving variational inequalities beyond monotonicity in real Banach spaces

Continuous-Time Accelerated Methods via a Hybrid Control Lens

A convergent hybrid three-term conjugate gradient method with sufficient descent property for unconstrained optimization

Accelerated Almost-Sure Convergence Rates for Nonconvex Stochastic Gradient Descent using Stochastic Learning Rates

Accelerating Nesterov's Method for Strongly Convex Functions with Lipschitz Gradient

Optimal Adaptive and Accelerated Stochastic Gradient Descent

Accelerated Gradient Methods Combining Tikhonov Regularization with Geometric Damping Driven by the Hessian