Anne Rubbens,Julien M. Hendrickx
Abstract:We propose a novel approach to obtain interpolation constraints for a wide range of function classes, i.e. necessary and sufficient constraints that a set of points, functions values and (sub)gradients must satisfy to ensure the existence of a global function of the class considered, consistent with this set. The derivation of such constraints is crucial for instance in the performance analysis of optimization methods, since obtaining a priori tight performance guarantees requires using a tight description of function classes of interest. Our method allows setting aside all analytic properties of the function class to work only at an algebraic level, and to easily obtain counterexamples when a condition characterizing a function class cannot serve as an interpolation constraint. As an illustration, we provide interpolation constraints for a class of non convex non smooth functions: weakly convex functions with bounded subgradients, and rely on these new interpolation constraints to outperform state of the art bounds on the performance of the subgradient method on this class.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is how to obtain an exact discrete definition of function classes in the performance analysis of optimization methods, especially for non - convex and non - smooth function classes. Specifically, the paper focuses on how to derive interpolation constraints, which can ensure that a given set of points, function values and (sub) - gradients are consistent with functions in a certain global function class. Such constraints are crucial for obtaining prior compact performance guarantees of optimization methods, because this requires the use of a compact description of the function class. However, the existing discretization methods are often not accurate enough, resulting in overly conservative performance analysis. Therefore, the paper proposes a new constraint - based method to obtain more accurate interpolation constraints, thereby improving the performance analysis of existing methods.
### Background and Motivation
In the performance analysis of optimization methods, it is usually necessary to operate on a certain class of functions or access a limited set of points, function values and (sub) - gradients, and require that these data are consistent with functions in a certain globally defined function class. For example, when analyzing the worst - case performance of an optimization method, it is necessary to derive bounds of certain performance metrics, which are applicable to any initial point satisfying the initial conditions and any element in a given function class. In order to obtain as tight bounds as possible, a compact description of the function class must be used. The traditional approach is to obtain these descriptions by discretizing global characteristics, but this method is often not accurate enough, resulting in overly conservative analysis results.
### Main Contributions of the Paper
1. **Constraint - Centered Method**:
- A new method is proposed to select from a class of constraints those that can define meaningful function classes and are interpolation constraints. This method avoids the difficulty of guessing interpolation constraints in traditional methods.
2. **New Tools for Evaluating Interpolability**:
- The concept of point - wise extensibility is introduced, that is, under certain continuity assumptions, if a constraint can be extended at any new point, then the constraint is an interpolation constraint. This tool simplifies the analysis of interpolability into a pure algebraic problem and simplifies the proof process.
3. **Compact Description of Quadratic Function Classes**:
- By applying the constraint - centered method and combining point - wise extensibility, a new interpolation constraint is obtained to describe a class of quadratic functions.
4. **Method for Automatically Testing Constraint Interpolability**:
- A numerical heuristic algorithm is proposed to calculate counterexamples of discretized descriptions, thereby automatically excluding non - compact descriptions.
5. **Compact Description of Weakly Convex Function Classes**:
- Compact descriptions of weakly convex function classes are provided, including functions with bounded sub - gradients or satisfying sharpness conditions. These descriptions improve the performance guarantees of existing methods.
### Related Work
- **Automated Performance Analysis**: The concept of interpolation constraints was initially introduced in optimization by [2] to make the PEP framework (Performance Estimation Problem) more precise. The PEP framework obtains performance bounds of optimization methods by formalizing optimization problems to search for worst - case functions.
- **Extension of Continuous Functions**: Interpolation constraints can also be regarded as conditions to ensure that a certain property is extended outside its domain, and this concept has been widely studied after Whitney's work.
### Conclusion
By proposing new constraint - based methods and evaluation tools, the paper solves the problem of accurately obtaining interpolation constraints in the performance analysis of optimization methods. These methods not only simplify the verification process of interpolability, but also provide more accurate descriptions for non - convex and non - smooth function classes, thereby improving the performance analysis of optimization methods.