Abstract:The numerical solution of the generalized eigenvalue problem for a singular matrix pencil is challenging due to the discontinuity of its eigenvalues. Classically, such problems are addressed by first extracting the regular part through the staircase form and then applying a standard solver, such as the QZ algorithm, to that regular part. Recently, several novel approaches have been proposed to transform the singular pencil into a regular pencil by relatively simple randomized modifications. In this work, we analyze three such methods by Hochstenbach, Mehl, and Plestenjak that modify, project, or augment the pencil using random matrices. All three methods rely on the normal rank and do not alter the finite eigenvalues of the original pencil. We show that the eigenvalue condition numbers of the transformed pencils are unlikely to be much larger than the $\delta$-weak eigenvalue condition numbers, introduced by Lotz and Noferini, of the original pencil. This not only indicates favorable numerical stability but also reconfirms that these condition numbers are a reliable criterion for detecting simple finite eigenvalues. We also provide evidence that, from a numerical stability perspective, the use of complex instead of real random matrices is preferable even for real singular matrix pencils and real eigenvalues. As a side result, we provide sharp left tail bounds for a product of two independent random variables distributed with the generalized beta distribution of the first kind or Kumaraswamy distribution.

Eigenvalue Analysis on Singularity in RBF Networks

Stability and Bifurcation Analysis in a Class of Two-Neuron Networks with Resonant Bilinear Terms

Singularity degree of structured random matrices

Stability Analysis of Recurrent Multilayer Perceptrons:Lmi Approach

Universal Sharpness Dynamics in Neural Network Training: Fixed Point Analysis, Edge of Stability, and Route to Chaos

Does SGD really happen in tiny subspaces?

Residual Networks as Nonlinear Systems: Stability Analysis using Linearization

On network dynamical systems with a nilpotent singularity

Analysis of eigenvalue condition numbers for a class of randomized numerical methods for singular matrix pencils

Singular Values for ReLU Layers

Three Dimensional Nilpotent Singularity and Sil’nikov Bifurcation

Trajectory Alignment: Understanding the Edge of Stability Phenomenon via Bifurcation Theory

Stability and Spatiotemporal Bifurcations in Spatially Distributed Neural Networks with Nonlocal Delay

Mixed Local-Nonlocal Operators and Singularity: A Multiple-Solution Perspective

High dimensional analysis reveals conservative sharpening and a stochastic edge of stability

Regional stability conditions for recurrent neural network-based control systems

Multiplicity of Singular Values for Tensors

New Necessary and Sufficient Conditions for Absolute Stability of Neural Networks

Asymptotic behavior of positive singular solutions to fractional Hardy-H\'enon equations

Deep Learning is Singular, and That's Good

Deep Learning Is Singular, and That’s Good