Abstract:In a Hilbert framework, for general convex differentiable optimization, we consider accelerated gradient dynamics combining Tikhonov regularization with Hessian-driven damping. The temporal discretization of these dynamics leads to a new class of first-order optimization algorithms with favorable properties. The Tikhonov regularization parameter is assumed to tend to zero as time tends to infinity, which preserves equilibria. The presence of the Tikhonov regularization term induces a strong property of convexity which vanishes asymptotically. To take advantage of the fast convergence rates attached to the heavy ball method in the strongly convex case, we consider inertial dynamics where the viscous damping coefficient is proportional to the square root of the Tikhonov regularization parameter, and hence converges to zero. The geometric damping, controlled by the Hessian of the function to be minimized, induces attenuation of the oscillations. Under an appropriate setting of the parameters, based on Lyapunov's analysis, we show that the trajectories provide at the same time several remarkable properties: fast convergence of values, fast convergence of gradients towards zero, and strong convergence to the minimum norm minimizer. We show that the corresponding proximal algorithms share the same properties as continuous dynamics. The numerical illustrations confirm the results obtained. This study extends a previous paper by the authors regarding similar problems without the presence of Hessian driven damping.

An Accelerated Regularized Chebyshev-Halley Method for Unconstrained Optimization

A New Algorithm For Unconstrained Optimization Problem With The Form Of Sum Of Squares Minimization

Unified Acceleration of High-Order Algorithms under General Hölder Continuity

Towards Unified Acceleration of High-Order Algorithms under H\"{o}lder Continuity and Uniform Convexity

A Proximal-Gradient Method for Constrained Optimization

An improved Dai‐Liao‐style hybrid conjugate gradient‐based method for solving unconstrained nonconvex optimization and extension to constrained nonlinear monotone equations

Accelerated Quasi-Newton Proximal Extragradient: Faster Rate for Smooth Convex Optimization

A Universal Accelerated Primal–Dual Method for Convex Optimization Problems

On the acceleration of the Barzilai-Borwein method

Accelerated nonlinear primal-dual hybrid gradient methods with applications to supervised machine learning

Stochastic Newton Proximal Extragradient Method

A hybrid inexact regularized Newton and negative curvature method

Nonconvex and Nonsmooth Optimization with Generalized Orthogonality Constraints: an Approximate Augmented Lagrangian Method

A class of modified accelerated proximal gradient methods for nonsmooth and nonconvex minimization problems

A Feasible Level Proximal Point Method for Nonconvex Sparse Constrained Optimization

Deterministic and Stochastic Accelerated Gradient Method for Convex Semi-Infinite Optimization

A regularized Newton method for $\ell_q$-norm composite optimization problems

Efficient Hyper-parameter Optimization with Cubic Regularization.

Accelerated Gradient Methods Combining Tikhonov Regularization with Geometric Damping Driven by the Hessian

A linearly convergent method for solving high-order proximal operator

Newton-CG methods for nonconvex unconstrained optimization with Hölder continuous Hessian