Abstract:We consider stability issues in minimizing a continuous (probably parameterized, nonconvex and nonsmooth) real-valued function f . We call a point stationary if all its possible directional derivatives are nonnegative. In this work, we focus on two notions of stability on stationary points of f : parametric stability and convergence stability. Parametric considerations are widely studied in various ﬁelds, including smoothed analysis, numerical stability, condition numbers and sensitivity analysis for linear programming. Parametric stability asks whether minor perturbations on parameters lead to dramatic changes in the position and f value of a stationary point. Meanwhile, convergence stability indicates a non-escapable solution: Any point sequence iteratively produced by an optimization algorithm cannot escape from a neighborhood of a stationary point but gets close to it in the sense that such stationary points are stable to the precision parameter and algorithmic numerical errors. It turns out that these notions have deep connections to geometry theory. We show that parametric stability is linked to deformations of graphs of functions. On the other hand, convergence stability is concerned with area partitioning of the function domain. Utilizing these connections, we prove quite tight conditions of these two stability notions for a wide range of functions and optimization algorithms with small enough step sizes and precision parameters. These conditions are subtle in the sense that a slightly weaker function requirement goes to the opposite of primitive intuitions and leads to wrong conclusions. We present three applications of this theory. First, we provide a thorough analysis of parametric stability and convergence stability on the state-of-the-art approximation algorithm computing Nash equilibria in bi-matrix games, Tsaknakis-Spirakis (TS for

Complexity Analysis Based on Tuning the Viscosity Parameter of the Su-Boyd-Candès Inertial Gradient Dynamics

An SDE Perspective on Stochastic Inertial Gradient Dynamics with Time-Dependent Viscosity and Geometric Damping

Convergence of Inertial Dynamics Driven by Sums of Potential and Nonpotential Operators with Implicit Newton-Like Damping

Optimal convergence rate of inertial gradient system with flat geometries and perturbations

Optimal Convergence Rates for Damped Inertial Gradient Dynamics with Flat Geometries

Stochastic Inertial Dynamics Via Time Scaling and Averaging

Inertial Methods with Viscous and Hessian driven Damping for Non-Convex Optimization

Global Convergence of Langevin Dynamics Based Algorithms for Nonconvex Optimization

Fast convex optimization via closed-loop time scaling of gradient dynamics

A fast continuous time approach with time scaling for nonsmooth convex optimization

Tikhonov Regularization for Stochastic Non-Smooth Convex Optimization in Hilbert Spaces

Fast Convergence of Inertial Multiobjective Gradient-like Systems with Asymptotic Vanishing Damping

A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights

An asymptotic viscosity selection result for the regularized Newton dynamic

A novel interpretation of Nesterov's acceleration via variable step-size linear multistep methods

Comparative analysis of accelerated gradient algorithms for convex optimization: high and super resolution ODE approach

A unified Euler--Lagrange system for analyzing continuous-time accelerated gradient methods

Stability analysis of the Navier-Stokes velocity tracking problem with bang-bang controls

A Doubly Nonlinear Evolution System with Threshold Effects Associated with Dry Friction

A Tutorial on a Lyapunov-Based Approach to the Analysis of Iterative Optimization Algorithms

Characterizing Parametric and Convergence Stability in Nonconvex and Nonsmooth Optimizations: A Geometric Approach