Abstract:We introduce a monotone and a non-monotone variant of the proximal gradient method in which the quadratic term is diagonal but may be indefinite, and is safeguarded by a trust region. Our method is a special case of the proximal quasi-Newton trust-region method of Aravkin et al. (SIAM J Optim 32(2):900–929, 2022). We provide closed-form solution of the step computation in certain cases where the nonsmooth term is separable and the trust region is defined in the infinity norm, so that no iterative subproblem solver is required. Our analysis expands upon that of (Aravkin et al. in SIAM J Optim 32(2):900–929, 2022) by generalizing the trust-region approach to problems with bound constraints. We provide an efficient open-source implementation of our method, named TRDH, in the Julia language in which Hessians approximations are given by diagonal quasi-Newton updates. TRDH evaluates one standard proximal operator and one indefinite proximal operator per iteration. We also analyze and implement a variant named iTRDH that performs a single indefinite proximal operator evaluation per iteration. We establish that iTRDH enjoys the same asymptotic worst-case iteration complexity as TRDH. We report numerical experience on unconstrained and bound-constrained problems, where TRDH and iTRDH are used both as standalone and subproblem solvers. Our results illustrate that, as standalone solvers, TRDH and iTRDH improve upon the quadratic regularization method R2 of (Aravkin et al. in SIAM J Optim 32(2):900–929, 2022) but also sometimes upon their quasi-Newton trust-region method, referred to here as TR-R2, in terms of smooth objective value and gradient evaluations. On challenging nonnegative matrix factorization, binary classification and data fitting problems, TRDH and iTRDH used as subproblem solvers inside TR improve upon TR-R2 for at least one choice of diagonal approximation and memory parameter.

Proximal Gradient Algorithm with Trust Region Scheme on Riemannian Manifold

Riemannian Trust Region Methods for Minimization

A Decentralized Proximal Gradient Tracking Algorithm for Composite Optimization on Riemannian Manifolds

Riemannian Trust Region Methods for SC$^1$ Minimization

A Trust Region Proximal Gradient Method for Nonlinear Multi-objective Optimization Problems

Convergence and worst-case complexity of adaptive Riemannian trust-region methods for optimization on manifolds

An Inexact Riemannian Proximal Gradient Method

Proximal methods for structured nonsmooth optimization over Riemannian submanifolds

Riemannian Stochastic Proximal Gradient Methods for Nonsmooth Optimization over the Stiefel Manifold

Sub‐sampled adaptive trust region method on Riemannian manifolds

Nonsmooth nonconvex optimization on Riemannian manifolds via bundle trust region algorithm

A TRUST-REGION METHOD FOR NONSMOOTH NONCONVEX OPTIMIZATION

The Proximal Gradient Method for Composite Optimization Problems on Riemannian Manifolds

Decentralized projected Riemannian gradient method for smooth optimization on compact submanifolds

A Riemannian Proximal Newton Method

Nonsmooth Optimization over the Stiefel Manifold and Beyond: Proximal Gradient Method and Recent Variants

A Proximal-Gradient Method for Constrained Optimization

Riemannian proximal gradient methods

A Riemannian Subspace BFGS Trust Region Method

The indefinite proximal gradient method

A Minibatch Proximal Stochastic Recursive Gradient Algorithm Using a Trust-Region-Like Scheme and Barzilai–Borwein Stepsizes