Abstract:We consider stability issues in minimizing a continuous (probably parameterized, nonconvex and nonsmooth) real-valued function f . We call a point stationary if all its possible directional derivatives are nonnegative. In this work, we focus on two notions of stability on stationary points of f : parametric stability and convergence stability. Parametric considerations are widely studied in various ﬁelds, including smoothed analysis, numerical stability, condition numbers and sensitivity analysis for linear programming. Parametric stability asks whether minor perturbations on parameters lead to dramatic changes in the position and f value of a stationary point. Meanwhile, convergence stability indicates a non-escapable solution: Any point sequence iteratively produced by an optimization algorithm cannot escape from a neighborhood of a stationary point but gets close to it in the sense that such stationary points are stable to the precision parameter and algorithmic numerical errors. It turns out that these notions have deep connections to geometry theory. We show that parametric stability is linked to deformations of graphs of functions. On the other hand, convergence stability is concerned with area partitioning of the function domain. Utilizing these connections, we prove quite tight conditions of these two stability notions for a wide range of functions and optimization algorithms with small enough step sizes and precision parameters. These conditions are subtle in the sense that a slightly weaker function requirement goes to the opposite of primitive intuitions and leads to wrong conclusions. We present three applications of this theory. First, we provide a thorough analysis of parametric stability and convergence stability on the state-of-the-art approximation algorithm computing Nash equilibria in bi-matrix games, Tsaknakis-Spirakis (TS for

Perturbation Analysis of the Euclidean Distance Matrix Optimization Problem and Its Numerical Implications.

Riemannian Optimization for Non-convex Euclidean Distance Geometry with Global Recovery Guarantees

On the local and global minimizers of the smooth stress function in Euclidean Distance Matrix problems

ADMM for Nonsmooth Composite Optimization under Orthogonality Constraints

Global Optimization with Orthogonality Constraints Via Stochastic Diffusion on Manifold

Maximizing Perturbation Radii for Robust Convex Quadratically Constrained Quadratic Programs

A Bregman-Style Improved ADMM and its Linearized Version in the Nonconvex Setting: Convergence and Rate Analyses

On a Unified and Simplified Proof for the Ergodic Convergence Rates of PPM, PDHG and ADMM

Characterizing Parametric and Convergence Stability in Nonconvex and Nonsmooth Optimizations: A Geometric Approach

A Feasible Method for Optimization with Orthogonality Constraints

A Riemannian ADMM

Convergence and Complexity Guarantee for Inexact First-order Riemannian Optimization Algorithms

Privacy-Preserving Distributed ADMM With Event-Triggered Communication

Understanding the ADMM Algorithm via High-Resolution Differential Equations

On the Efficiency of Random Permutation for ADMM and Coordinate Descent

An Improved Analysis of Semidefinite Approximation Bound for Nonconvex Nonhomogeneous Quadratic Optimization with Ellipsoid Constraints

Perturbation analysis of a class of composite optimization problems

Linear Convergence of the Alternating Direction Method of Multipliers for a Class of Convex Optimization Problems

Linearized ADMM for Nonconvex Nonsmooth Optimization with Convergence Analysis

Efficient Semidefinite Programming with approximate ADMM

Convergence Analysis of Generalized ADMM with Majorization for Linearly Constrained Composite Convex Optimization