Abstract:We consider stability issues in minimizing a continuous (probably parameterized, nonconvex and nonsmooth) real-valued function f . We call a point stationary if all its possible directional derivatives are nonnegative. In this work, we focus on two notions of stability on stationary points of f : parametric stability and convergence stability. Parametric considerations are widely studied in various ﬁelds, including smoothed analysis, numerical stability, condition numbers and sensitivity analysis for linear programming. Parametric stability asks whether minor perturbations on parameters lead to dramatic changes in the position and f value of a stationary point. Meanwhile, convergence stability indicates a non-escapable solution: Any point sequence iteratively produced by an optimization algorithm cannot escape from a neighborhood of a stationary point but gets close to it in the sense that such stationary points are stable to the precision parameter and algorithmic numerical errors. It turns out that these notions have deep connections to geometry theory. We show that parametric stability is linked to deformations of graphs of functions. On the other hand, convergence stability is concerned with area partitioning of the function domain. Utilizing these connections, we prove quite tight conditions of these two stability notions for a wide range of functions and optimization algorithms with small enough step sizes and precision parameters. These conditions are subtle in the sense that a slightly weaker function requirement goes to the opposite of primitive intuitions and leads to wrong conclusions. We present three applications of this theory. First, we provide a thorough analysis of parametric stability and convergence stability on the state-of-the-art approximation algorithm computing Nash equilibria in bi-matrix games, Tsaknakis-Spirakis (TS for

A GENERALIZED PERSIDSKII THEOREM AND ITS APPLICATIONS TO NONSMOOTH GRADIENT DYNAMICAL SYSTEMS

A new approach to almost-sure asymptotic stability of stochastic systems of higher dimension

A dynamic gradient approach to Pareto optimization with nonsmooth convex objective functions

Convex and Non-convex Optimization Under Generalized Smoothness

An SDE Perspective on Stochastic Inertial Gradient Dynamics with Time-Dependent Viscosity and Geometric Damping

Convergence of Constant Step Stochastic Gradient Descent for Non-Smooth Non-Convex Functions

Support Vector Classifiers Via Gradient Systems with Discontinuous Righthand Sides

A semismooth conjugate gradients method – theoretical analysis

Fast Convergence of Inertial Multiobjective Gradient-like Systems with Asymptotic Vanishing Damping

Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations

Characterizing Parametric and Convergence Stability in Nonconvex and Nonsmooth Optimizations: A Geometric Approach

Convergence Analysis of a Class of Nonsmooth Gradient Systems.

Third Order Dynamical Systems for the Sum of Two Generalized Monotone Operators

A geometric integration approach to smooth optimisation: Foundations of the discrete gradient method

Asymptotic Analysis via Stochastic Differential Equations of Gradient Descent Algorithms in Statistical and Computational Paradigms

Nonsmooth Analysis of Doubly Nonlinear Second-Order Evolution Equations with Non-Convex Energy Functionals

Generalized Gradient Flows with Provable Fixed-Time Convergence and Fast Evasion of Non-Degenerate Saddle Points

Directional Smoothness and Gradient Methods: Convergence and Adaptivity

Continuous Dynamics Related to Monotone Inclusions and Non-Smooth Optimization Problems

Generalized Metric Subregularity with Applications to High-Order Regularized Newton Methods

UNIFORM POLYNOMIAL STABILITY OF SECOND ORDER INTEGRO-DIFFERENTIAL EQUATIONS IN HILBERT SPACES WITH POSITIVE DEFINITE KERNELS