Abstract:We consider stability issues in minimizing a continuous (probably parameterized, nonconvex and nonsmooth) real-valued function f . We call a point stationary if all its possible directional derivatives are nonnegative. In this work, we focus on two notions of stability on stationary points of f : parametric stability and convergence stability. Parametric considerations are widely studied in various ﬁelds, including smoothed analysis, numerical stability, condition numbers and sensitivity analysis for linear programming. Parametric stability asks whether minor perturbations on parameters lead to dramatic changes in the position and f value of a stationary point. Meanwhile, convergence stability indicates a non-escapable solution: Any point sequence iteratively produced by an optimization algorithm cannot escape from a neighborhood of a stationary point but gets close to it in the sense that such stationary points are stable to the precision parameter and algorithmic numerical errors. It turns out that these notions have deep connections to geometry theory. We show that parametric stability is linked to deformations of graphs of functions. On the other hand, convergence stability is concerned with area partitioning of the function domain. Utilizing these connections, we prove quite tight conditions of these two stability notions for a wide range of functions and optimization algorithms with small enough step sizes and precision parameters. These conditions are subtle in the sense that a slightly weaker function requirement goes to the opposite of primitive intuitions and leads to wrong conclusions. We present three applications of this theory. First, we provide a thorough analysis of parametric stability and convergence stability on the state-of-the-art approximation algorithm computing Nash equilibria in bi-matrix games, Tsaknakis-Spirakis (TS for

Characterization of optimization problems that are solvable iteratively with linear convergence

Linear Convergence of Subgradient Algorithm for Convex Feasibility on Riemannian Manifolds

Linearization Algorithms for Fully Composite Optimization

Convergence of Inexact Steepest Descent Algorithm for Multiobjective Optimizations on Riemannian Manifolds Without Curvature Constraints

Convergence Analysis of Gradient Algorithms on Riemannian Manifolds Without Curvature Constraints and Application to Riemannian Mass

Linear Convergence of Stochastic Iterative Greedy Algorithms With Sparse Constraints

Linear convergence of first order methods for non-strongly convex optimization

Characterizing Parametric and Convergence Stability in Nonconvex and Nonsmooth Optimizations: A Geometric Approach

Linear convergence of forward-backward accelerated algorithms without knowledge of the modulus of strong convexity

Convergence and Complexity Guarantee for Inexact First-order Riemannian Optimization Algorithms

Linear-memory and Decomposition-invariant Linearly Convergent Conditional Gradient Algorithm for Structured Polytopes

Convergence of successive linear programming algorithms for noisy functions

A Linearly Convergent Conditional Gradient Algorithm with Applications to Online and Stochastic Optimization

Learning to optimize with convergence guarantees using nonlinear system theory

On Linear Convergence in Smooth Convex-Concave Bilinearly-Coupled Saddle-Point Optimization: Lower Bounds and Optimal Algorithms

Linear convergence of accelerated conditional gradient algorithms in spaces of measures

Geodesic Convexity of the Symmetric Eigenvalue Problem and Convergence of Steepest Descent

Geodesic Convexity of the Symmetric Eigenvalue Problem and Convergence of Riemannian Steepest Descent

Variational Analysis Perspective on Linear Convergence of Some First Order Methods for Nonsmooth Convex Optimization Problems

On the linear convergence rate of Riemannian proximal gradient method