Abstract:In this paper we consider minimization of a difference-of-convex (DC) function with and without linear constraints. We first study a smooth approximation of a generic DC function, termed difference-of-Moreau-envelopes (DME) smoothing, where both components of the DC function are replaced by their respective Moreau envelopes. The resulting smooth approximation is shown to be Lipschitz differentiable, capture stationary points, local, and global minima of the original DC function, and enjoy some growth conditions, such as level-boundedness and coercivity, for broad classes of DC functions. We then develop four algorithms for solving DC programs with and without linear constraints based on the DME smoothing. In particular, for a smoothed DC program without linear constraints, we show that the classic gradient descent method as well as an inexact variant can obtain a stationary solution in the limit with a convergence rate of $\mathcal{O}(K^{-1/2})$, where $K$ is the number of proximal evaluations of both components. Furthermore, when the DC program is explicitly constrained in an affine subspace, we combine the smoothing technique with the augmented Lagrangian function and derive two variants of the augmented Lagrangian method (ALM), named LCDC-ALM and composite LCDC-ALM, focusing on different structures of the DC objective function. We show that both algorithms find an $\epsilon$-approximate stationary solution of the original DC program in $\mathcal{O}(\epsilon^{-2})$ iterations. Comparing to existing methods designed for linearly constrained weakly convex minimization, the proposed ALM-based algorithms can be applied to a broader class of problems, where the objective contains a nonsmooth concave component. Finally, numerical experiments are presented to demonstrate the performance of the proposed algorithms.

Local Convergence Analysis of the Inexact Smoothing Levenberg-Marquardt Algorithm

Local Convergence of Inexact Methods under the Hölder Condition

Local and Global Behavior for Algorithms of Solving Equations

Convergence rate of inexact proximal point algorithms for operator with holder metric subregularity

A Smoothing Levenberg-Marquardt Type Method for Lcp

Local convergence analysis of an inexact trust-region method for nonsmooth optimization

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Convergence analysis of inexact proximal point algorithms on Hadamard manifolds

Convergence of Inexact Steepest Descent Algorithm for Multiobjective Optimizations on Riemannian Manifolds Without Curvature Constraints

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

On the local convergence of the semismooth Newton method for composite optimization

Globalization of convergence of the constrained piecewise Levenberg–Marquardt method

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

A New Adaptive Levenberg–Marquardt Method for Nonlinear Equations and Its Convergence Rate under the Hölderian Local Error Bound Condition

Local convergence of an inexact proximal algorithm for weakly convex functions

Convergence Behavior of Gauss-Newton's Method and Extensions of the Smale Point Estimate Theory.

Convergence analysis of Levenberg-Marquardt method with Singular Scaling for nonzero residue nonlinear least-squares problems

Algorithms for Difference-of-Convex (DC) Programs Based on Difference-of-Moreau-Envelopes Smoothing

On the Local Convergence of a Stochastic Semismooth Newton Method for Nonsmooth Nonconvex Optimization

Inexact Proximal Cubic Regularized Newton Methods for Convex Optimization