Abstract: We study the smooth minimax optimization problem $\min_{\bf x}\max_{\bf y} f({\bf x},{\bf y})$, where $f$ is $\ell$-smooth, strongly-concave in ${\bf y}$ but possibly nonconvex in ${\bf x}$. Most of existing works focus on finding the first-order stationary points of the function $f({\bf x},{\bf y})$ or its primal function $P({\bf x})\triangleq \max_{\bf y} f({\bf x},{\bf y})$, but few of them focus on achieving second-order stationary points. In this paper, we propose a novel approach for minimax optimization, called Minimax Cubic Newton (MCN), which could find an $\big(\varepsilon,\kappa^{1.5}\sqrt{\rho\varepsilon}\,\big)$-second-order stationary point of $P({\bf x})$ with calling ${\mathcal O}\big(\kappa^{1.5}\sqrt{\rho}\varepsilon^{-1.5}\big)$ times of second-order oracles and $\tilde{\mathcal O}\big(\kappa^{2}\sqrt{\rho}\varepsilon^{-1.5}\big)$ times of first-order oracles, where $\kappa$ is the condition number and $\rho$ is the Lipschitz continuous constant for the Hessian of $f({\bf x},{\bf y})$. In addition, we propose an inexact variant of MCN for high-dimensional problems to avoid calling expensive second-order oracles. Instead, our method solves the cubic sub-problem inexactly via gradient descent and matrix Chebyshev expansion. This strategy still obtains the desired approximate second-order stationary point with high probability but only requires $\tilde{\mathcal O}\big(\kappa^{1.5}\ell\varepsilon^{-2}\big)$ Hessian-vector oracle calls and $\tilde{\mathcal O}\big(\kappa^{2}\sqrt{\rho}\varepsilon^{-1.5}\big)$ first-order oracle calls. To the best of our knowledge, this is the first work that considers the non-asymptotic convergence behavior of finding second-order stationary points for minimax problems without the convex-concave assumptions.

Sequential M-Stationarity Conditions for General Optimization Problems

First-order Optimality Conditions for Two Classes of Generalized Nonsmooth Semi-Infinite Optimization

Stationarity in nonsmooth optimization between geometrical motivation and topological relevance

Mordukhovich Stationarity for Mathematical Programs with Switching Constraints under Weak Constraint Qualifications

Stochastic Successive Convex Approximation for Non-Convex Constrained Stochastic Optimization

A sequential optimality condition for Mathematical Programs with Cardinality Constraints

Solving Nonsmooth Nonconvex Compound Stochastic Programs with Applications to Risk Measure Minimization

The sparse(st) optimization problem: reformulations, optimality, stationarity, and numerical results

Cardinality-Constrained Multi-Objective Optimization: Novel Optimality Conditions and Algorithms

First- and second-order optimality conditions of nonsmooth sparsity multiobjective optimization via variational analysis

Finding Second-Order Stationary Points in Nonconvex-Strongly-Concave Minimax Optimization

Riemannian Smoothing Gradient Type Algorithms]{Riemannian Smoothing Gradient Type Algorithms for Nonsmooth Optimization Problem on Compact Riemannian Submanifold Embedded in Euclidean Space

On the directional asymptotic approach in optimization theory

Almost-sure convergence of iterates and multipliers in stochastic sequential quadratic optimization

Goldstein Stationarity in Lipschitz Constrained Optimization

Numerical Methods for Convex Multistage Stochastic Optimization

The Minimization of Piecewise Functions: Pseudo Stationarity

A note on stationarity in constrained optimization

Globally Convergent Algorithm for Solving Stationary Points for Mathematical Programs with Complementarity Constraints Via Nonsmooth Reformulations

On Enhanced KKT Optimality Conditions for Smooth Nonlinear Optimization