Abstract:In this paper, we address a nonconvexly constrained nonsmooth optimization problem involving the composition of a weakly convex function and a smooth mapping. To find a stationary point of the target problem, we propose a variable smoothing-type algorithm by combining the ideas of (i) translating the constrained problem into a Euclidean optimization problem with a smooth parametrization of the constraint set; (ii) exploiting a sequence of smoothed surrogate functions, of the cost function, given with the Moreau envelope of a weakly convex function. The proposed algorithm produces a vector sequence by the gradient descent update of a smoothed surrogate function at each iteration. In a case where the proximity operator of the weakly convex function is available, the proposed algorithm does not require any iterative solver for subproblems therein. By leveraging tools in the variational analysis, we show the so-called {\em gradient consistency property}, which is a key ingredient for smoothing-type algorithms, of the smoothed surrogate function used in this paper. Based on the gradient consistency property, we also establish an asymptotic convergence analysis for the proposed algorithm. Numerical experiments demonstrate the efficacy of the proposed algorithm.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the weakly convex composite minimization problem with non - convex constraints. Specifically, the authors focus on a non - smooth optimization problem in the Euclidean space \(X\), whose objective function is composed of a weakly convex function and a smooth mapping, and has a non - convex constraint set \(C\subset X\). ### Problem Description Consider the following non - smooth optimization problem with non - convex constraints: \[ \text{Problem 1.1:} \quad \min_{x\in C}(h(x)+g(S(x))) \] where: - \(X\) and \(Z\) are Euclidean spaces (i.e., finite - dimensional real Hilbert spaces). - \(C\subset X\) is a possibly non - convex closed set. - \(h:X\rightarrow\mathbb{R}\) is continuously differentiable, and its gradient \(\nabla h\) is Lipschitz continuous on \(C\). - \(S:X\rightarrow Z\) is a continuously differentiable (possibly non - linear) mapping. - \(g:Z\rightarrow\mathbb{R}\) is a possibly non - smooth Lipschitz continuous function and is \(\eta\)-weakly convex (i.e., \(g +\frac{\eta}{2}\|\cdot\|^{2}\) is convex). ### Solution To find the stable points of the above problem, the authors propose a variational smoothing algorithm, which combines the following two ideas: 1. **Parameterize the constraint set**: Transform the constrained problem into a Euclidean optimization problem with smooth parameterization. 2. **Smoothing surrogate function**: Use the Moreau envelope to smooth the objective function. The specific steps are as follows: - Use the Moreau envelope to construct a series of smoothing surrogate functions. - Update the smoothing surrogate function using gradient descent in each iteration. - If the proximity operator of the weakly convex function is available, there is no need for an iterative solver in the sub - problem. Through variational analysis tools, the authors prove the gradient consistency property of the used smoothing surrogate functions and establish the asymptotic convergence analysis of the algorithm based on this. ### Numerical Experiments Numerical experiments verify the effectiveness of this algorithm in sparse spectral clustering and sparse principal component analysis. ### Summary This paper aims to solve the weakly convex composite minimization problem with non - convex constraints, proposes a variational smoothing algorithm that combines parameterization and smoothing techniques, and proves its effectiveness and convergence.

A Variable Smoothing for Weakly Convex Composite Minimization with Nonconvex Constraint

A Proximal Variable Smoothing for Nonsmooth Minimization Involving Weakly Convex Composite with MIMO Application

A variable smoothing for Nonconvexly constrained nonsmooth optimization with application to sparse spectral clustering

A Smoothing Stochastic Gradient Method for Composite Optimization

Constrained Global Optimization by Smoothing

Self-concordant Smoothing for Large-Scale Convex Composite Optimization

Adaptive smoothing mini-batch stochastic accelerated gradient method for nonsmooth convex stochastic composite optimization

Stochastic smoothing accelerated gradient method for nonsmooth convex composite optimization

A Smoothing Direct Search Method for Monte Carlo-Based Bound Constrained Composite Nonsmooth Optimization.

A NONMOTOTONE ALGORITHM FOR MINIMIZING NONSMOOTH COMPOSITE FUNCTIONS

A feasible smoothing accelerated projected gradient method for nonsmooth convex optimization

Variance-Reduced Proximal Stochastic Gradient Descent for Non-convex Composite optimization.

Mini-batch stochastic Nesterov's smoothing method for constrained convex stochastic composite optimization

Nonsmooth, Nonconvex Optimization Using Functional Encoding and Component Transition Information

A VMiPG method for composite optimization with nonsmooth term having no closed-form proximal mapping

Smooth over-parameterized solvers for non-smooth structured optimization

A successive difference-of-convex approximation method for a class of nonconvex nonsmooth optimization problems

A Conditional Gradient Framework for Composite Convex Minimization with Applications to Semidefinite Programming

A Nonlinear Bregman Primal-Dual Framework for Optimizing Nonconvex Infimal Convolutions

A Unified Inexact Stochastic ADMM for Composite Nonconvex and Nonsmooth Optimization

A Note on Complexity for Two Classes of Structured Non-Smooth Non-Convex Compositional Optimization