Abstract:SIAM Journal on Optimization, Volume 34, Issue 1, Page 1006-1044, March 2024. Minimax problems have recently attracted a lot of research interests. A few efforts have been made to solve decentralized nonconvex strongly-concave (NCSC) minimax-structured optimization; however, all of them focus on smooth problems with at most a constraint on the maximization variable. In this paper, we make the first attempt on solving composite NCSC minimax problems that can have convex nonsmooth terms on both minimization and maximization variables. Our algorithm is designed based on a novel reformulation of the decentralized minimax problem that introduces a multiplier to absorb the dual consensus constraint. The removal of dual consensus constraint enables the most aggressive (i.e., local maximization instead of a gradient ascent step) dual update that leads to the benefit of taking a larger primal stepsize and better complexity results. In addition, the decoupling of the nonsmoothness and consensus on the dual variable eases the analysis of a decentralized algorithm; thus our reformulation creates a new way for interested researchers to design new (and possibly more efficient) decentralized methods on solving NCSC minimax problems. We show a global convergence result of the proposed algorithm and an iteration complexity result to produce a (near) stationary point of the reformulation. Moreover, a relation is established between the (near) stationarities of the reformulation and the original formulation. With this relation, we show that when the dual regularizer is smooth, our algorithm can have lower complexity results (with reduced dependence on a condition number) than existing ones to produce a near-stationary point of the original formulation. Numerical experiments are conducted on a distributionally robust logistic regression to demonstrate the performance of the proposed algorithm.

A template for gradient norm minimization

Gradient Methods with Memory for Minimizing Composite Functions

Complementary Composite Minimization, Small Gradients in General Norms, and Applications

Parameter-Free Accelerated Gradient Descent for Nonconvex Minimization

Adaptive first-order methods with enhanced worst-case rates

Optimal and parameter-free gradient minimization methods for convex and nonconvex optimization

Gradient Methods with Online Scaling

A class of modified accelerated proximal gradient methods for nonsmooth and nonconvex minimization problems

A Proximal-Gradient Method for Constrained Optimization

A Smooth Primal-Dual Optimization Framework for Nonsmooth Composite Convex Minimization

Universal Gradient Descent Ascent Method for Nonconvex-Nonconcave Minimax Optimization

On a Family of Relaxed Gradient Descent Methods for Quadratic Minimization

A Conditional Gradient Framework for Composite Convex Minimization with Applications to Semidefinite Programming

An Adaptive Stochastic Gradient Method with Non-negative Gauss-Newton Stepsizes

Decentralized Gradient Descent Maximization Method for Composite Nonconvex Strongly-Concave Minimax Problems

Accelerated Objective Gap and Gradient Norm Convergence for Gradient Descent via Long Steps

Gradient descent with adaptive stepsize converges (nearly) linearly under fourth-order growth

Accelerated Gradient Algorithms with Adaptive Subspace Search for Instance-Faster Optimization

Reparametrizing gradient descent

Stagewise Accelerated Stochastic Gradient Methods for Nonconvex Optimization