Abstract:Minimax problems have recently attracted a lot of research interests. A few efforts have been made to solve decentralized nonconvex strongly-concave (NCSC) minimax-structured optimization; however, all of them focus on smooth problems with at most a constraint on the maximization variable. In this paper, we make the first attempt on solving composite NCSC minimax problems that can have convex nonsmooth terms on both minimization and maximization variables. Our algorithm is designed based on a novel reformulation of the decentralized minimax problem that introduces a multiplier to absorb the dual consensus constraint. The removal of dual consensus constraint enables the most aggressive (i.e., local maximization instead of a gradient ascent step) dual update that leads to the benefit of taking a larger primal stepsize and better complexity results. In addition, the decoupling of the nonsmoothness and consensus on the dual variable eases the analysis of a decentralized algorithm; thus our reformulation creates a new way for interested researchers to design new (and possibly more efficient) decentralized methods on solving NCSC minimax problems. We show a global convergence result of the proposed algorithm and an iteration complexity result to produce a (near) stationary point of the reformulation. Moreover, a relation is established between the (near) stationarities of the reformulation and the original formulation. With this relation, we show that when the dual regularizer is smooth, our algorithm can have lower complexity results (with reduced dependence on a condition number) than existing ones to produce a near-stationary point of the original formulation. Numerical experiments are conducted on a distributionally robust logistic regression to demonstrate the performance of the proposed algorithm.

Extended Levenberg-Marquardt Method for Composite Function Minimization

An Enhanced Levenberg--Marquardt Method via Gram Reduction

Extended Newton Methods for Multiobjective Optimization: Majorizing Function Technique and Convergence Analysis

A proximal method for composite minimization

The Modified Levenberg-Marquardt Method for Nonlinear Equations with Cubic Convergence

Majorizing Functions and Convergence of the Gauss-Newton Method for Convex Composite Optimization

An Adaptive Multi-step Levenberg–Marquardt Method

Gauss–Newton Method for Convex Composite Optimizations on Riemannian Manifolds

A Modified Mnemonic Enhancement Optimization Method For Solving Parametric Nonlinear Programming Problems

Convergence of Inexact Steepest Descent Algorithm for Multiobjective Optimizations on Riemannian Manifolds Without Curvature Constraints

A Smoothing Levenberg-Marquardt Type Method for Lcp

On the Levenberg-Marquardt Methods for Convex Constrained Nonlinear Equations

Decentralized gradient descent maximization method for composite nonconvex strongly-concave minimax problems

A new linearization technique for minimax linear fractional programming

Quadratic Convergence of Levenberg-Marquardt Method for Elliptic and Parabolic Inverse Robin Problems

Relaxed Majorization-Minimization for Non-Smooth and Non-Convex Optimization

A Shamanskii-like Levenberg-Marquardt Method for Nonlinear Equations

A Minimization Approach for Minimax Optimization with Coupled Constraints

Bregman Proximal Linearized ADMM for Minimizing Separable Sums Coupled by a Difference of Functions