Abstract:There is growing interest in nonconvex minimax problems that is driven by an abundance of applications. Our focus is on nonsmooth, nonconvex-strongly concave minimax, thus departing from the more common weakly convex and smooth models assumed in the recent literature. We present proximal gradient schemes with either parallel or alternating steps. We show that both methods can be analyzed through a single scheme within a unified analysis that relies on expanding a general convergence mechanism used for analyzing nonconvex, nonsmooth optimization problems. In contrast to the current literature, which focuses on the complexity of obtaining nearly approximate stationary solutions, we prove subsequence convergence to a critical point of the primal objective and global convergence when the latter is semialgebraic. Furthermore, the complexity results we provide are with respect to approximate stationary solutions. Lastly, we expand the scope of problems that can be addressed by generalizing one of the steps with a Bregman proximal gradient update, and together with a few adjustments to the analysis, this allows us to extend the convergence and complexity results to this broader setting. Funding: The research of E. Cohen was partially supported by a doctoral fellowship from the Israel Science Foundation [Grant 2619-20] and Deutsche Forschungsgemeinschaft [Grant 800240]. The research of M. Teboulle was partially supported by the Israel Science Foundation [Grant 2619-20] and Deutsche Forschungsgemeinschaft [Grant 800240].

The proximal alternating direction method of multipliers in the nonconvex setting: convergence analysis and rates

A Proximal Alternating Linearization Method for Nonconvex Optimization Problems

Local Linear Convergence of the Alternating Direction Method of Multipliers for Nonconvex Separable Optimization Problems

A Homotopy Alternating Direction Method of Multipliers for Linearly Constrained Separable Convex Optimization

A Proximal Alternating Direction Method of Multipliers for a Minimization Problem with Nonconvex Constraints

A Hybrid Bregman Alternating Direction Method of Multipliers for the Linearly Constrained Difference-of-convex Problems

On the Convergence of a Generalized Alternating Direction Method of Multipliers with Semi-Proximal Terms

An Inertial Proximal Alternating Direction Method of Multipliers for Nonconvex Optimization.

Local Convergence Properties of Douglas–Rachford and Alternating Direction Method of Multipliers

Convergence of alternating direction method for minimizing sum of two nonconvex functions with linear constraints

Alternating Direction Method of Multipliers with Variable Metric Indefinite Proximal Terms for Convex Optimization

A Proximal Point Analysis of the Preconditioned Alternating Direction Method of Multipliers.

A Proximal Alternating Linearization Method for Minimizing the Sum of Two Convex Functions

Alternating Direction Method of Multipliers for a Class of Nonconvex Bilinear Optimization: Convergence Analysis and Applications.

On convergence rates of proximal alternating direction method of multipliers

Alternating and Parallel Proximal Gradient Methods for Nonsmooth, Nonconvex Minimax: A Unified Convergence Analysis

Variants of Alternating Minimization Method with Sublinear Rates of Convergence for Convex Optimization

N ov 2 01 6 A Proximal Stochastic Quasi-Newton Algorithm

Semi-Proximal Point Method for Nonsmooth Convex-Concave Minimax Optimization

Global Convergence of Splitting Methods for Nonconvex Composite Optimization

Iteratively Linearized Reweighted Alternating Direction Method of Multipliers for a Class of Nonconvex Problems