Abstract:Nonconvex and nonsmooth optimization problems are frequently encountered in much of statistics, business, science and engineering, but they are not yet widely recognized as a technology in the sense of scalability. A reason for this relatively low degree of popularity is the lack of a well developed system of theory and algorithms to support the applications, as is the case for its convex counterpart. This paper aims to take one step in the direction of disciplined nonconvex and nonsmooth optimization. In particular, we consider in this paper some constrained nonconvex optimization models in block decision variables, with or without coupled affine constraints. In the absence of coupled constraints, we show a sublinear rate of convergence to an \(\epsilon \)-stationary solution in the form of variational inequality for a generalized conditional gradient method, where the convergence rate is dependent on the Hölderian continuity of the gradient of the smooth part of the objective. For the model with coupled affine constraints, we introduce corresponding \(\epsilon \)-stationarity conditions, and apply two proximal-type variants of the ADMM to solve such a model, assuming the proximal ADMM updates can be implemented for all the block variables except for the last block, for which either a gradient step or a majorization–minimization step is implemented. We show an iteration complexity bound of \(O(1/\epsilon ^2)\) to reach an \(\epsilon \)-stationary solution for both algorithms. Moreover, we show that the same iteration complexity of a proximal BCD method follows immediately. Numerical results are provided to illustrate the efficacy of the proposed algorithms for tensor robust PCA and tensor sparse PCA problems.

A class of bilinear matrix constraint optimization problem and its applications

A Novel Method for a Class of Structured Low-Rank Minimizations with Equality Constraint.

A successive convex optimization method for bilinear matrix inequality problems and its application to static output‐feedback control

Low nonconvexity-rank bilinear matrix inequalities: algorithms and applications in robust controller and structure designs

Alternating Direction Method of Multipliers for a Class of Nonconvex Bilinear Optimization: Convergence Analysis and Applications.

Indefinite Linearized Augmented Lagrangian Method for Convex Programming with Linear Inequality Constraints

A Smoothing Augmented Lagrangian Method for Nonconvex, Nonsmooth Constrained Programs and Its Applications to Bilevel Problems

An Extension of Analytic Solutions of a Class of Constrained Matrix Minimization Problems.

General Constrained Matrix Optimization

Optimality Conditions and Numerical Algorithms for A Class of Linearly Constrained Minimax Optimization Problems

Nonconvex and Nonsmooth Optimization with Generalized Orthogonality Constraints: an Approximate Augmented Lagrangian Method

Linearized Alternating Direction Method of Multipliers for Constrained Nonconvex Regularized Optimization.

A Minimization Approach for Minimax Optimization with Coupled Constraints

Nonconvex Mixed Matrix Minimization

THE ANALYTIC SOLUTIONS OF A CLASS OF CONSTRAINED MATRIX MINIMIZATION AND MAXIMIZATION PROBLEMS WITH APPLICATIONS

AN OUTER APPROXIMATION METHOD FOR A CLASS OF MINIMAX CONVEX MINLP PROBLEMS

Structured Nonconvex and Nonsmooth Optimization: Algorithms and Iteration Complexity Analysis

Global optimization algorithm for mixed integer quadratically constrained quadratic program

Matrix Function Optimization Problems under Orthonormal Constraint

A Homotopy Alternating Direction Method of Multipliers for Linearly Constrained Separable Convex Optimization

Dynamic system methods for solving mixed linear matrix inequalities and linear vector inequalities and equalities