Abstract:Marginal optima are minima or maxima of a function with many nearly flat directions. In settings with many competing optima, marginal ones tend to attract algorithms and physical dynamics. Often, the important family of marginal attractors are a vanishing minority compared with nonmarginal optima and other unstable stationary points. We introduce a generic technique for conditioning the statistics of stationary points in random landscapes on their marginality, and apply it in three isotropic settings with qualitatively different structure: in the spherical spin-glasses, where the energy is Gaussian and its Hessian is GOE; in multispherical spin glasses, which are Gaussian but non-GOE; and in sums of squared spherical random functions, which are non-Gaussian. In these problems we are able to fully characterize the distribution of marginal optima in the landscape, including when they are in the minority.
What problem does this paper attempt to address?
### The problems the paper attempts to solve
What this paper attempts to solve is how to condition stationary points in a random landscape, especially those stationary points with marginal minima. Marginal minima refer to those minima with a pseudogap, that is, the spectrum of their Hessian matrix has a gap near zero. These stationary points are very important in many practical applications, such as in optimization algorithms and physical dynamics, but it is difficult for traditional methods to effectively perform conditional analysis on them.
### Specific problem background
1. **Stationary points in complex landscapes**:
- In many disciplines, such as glassy physics, spin glasses, statistical inference, etc., the study of rugged landscapes is very important.
- These systems usually contain a large number of local minima and saddle points, causing dynamic processes to easily get trapped in non - optimal solutions.
2. **The importance of threshold energy**:
- In mean - field theory, physical and algorithmic dynamics are usually considered to be stuck at a certain energy level, called threshold energy.
- The threshold energy is the transition point in the landscape from mainly containing saddle points to mainly containing minima.
- However, recent research has found that the threshold energy is not important for simple gradient - descent dynamics, and the dynamic process can reach states above or below the threshold energy.
3. **The universality of marginal minima**:
- Although marginal minima may be small in number, they are still often found in dynamic processes.
- This universality indicates that the distribution of marginal minima can be used to constrain non - equilibrium dynamic behavior.
4. **Limitations of traditional methods**:
- Traditional analysis methods are difficult to directly condition marginal minima, especially in non - Gaussian random landscapes.
- The Legendre transform using Parisi parameters can condition certain models, but this method is limited to marginal minima at specific energy levels.
### Main contributions of the paper
1. **Introduction of a general technique**:
- The paper introduces a general technique that can condition the marginality of stationary points in a random landscape.
- This technique is achieved by selecting the smallest eigenvalue of the matrix and further ensuring that this eigenvalue is on the edge of the continuous spectrum, thus forming a pseudogap.
2. **Application to specific models**:
- The author has applied this technique to three different models: spherical spin glasses, multi - spherical spin glasses, and the sum of squared random functions.
- In these models, the author was able to fully characterize the distribution of marginal minima in the landscape, even if they are in the minority in number.
### Conclusion
By introducing this new conditioning technique, the paper provides a powerful tool for studying marginal minima in complex landscapes. This not only helps to understand the role of these stationary points in dynamic processes but also provides a new perspective for optimization algorithms and physical dynamics.