Average polynomial time for eigenvector computations

Diego Armentano,Carlos Beltrán,Michael Shub

DOI: https://doi.org/10.48550/arXiv.1410.2179

2014-10-08

Numerical Analysis

Abstract:We describe two algorithms for the eigenvalue, eigenvector problem which, on input a Gaussian matrix with complex entries, finish with probability 1 and in average polynomial time.

What problem does this paper attempt to address?

Fast and backward stable computation of the eigenvalues and eigenvectors of matrix polynomials

Jared Aurentz,Thomas Mach,Leonardo Robol,Raf Vandebril,David S. Watkins

DOI: https://doi.org/10.48550/arXiv.1611.10142

2016-11-30

Numerical Analysis

Abstract:In the last decade matrix polynomials have been investigated with the primary focus on adequate linearizations and good scaling techniques for computing their eigenvalues and eigenvectors. In this article we propose a new method for computing a factored Schur form of the associated companion pencil. The algorithm has a quadratic cost in the degree of the polynomial and a cubic one in the size of the coefficient matrices. Also the eigenvectors can be computed at the same cost. The algorithm is a variant of Francis's implicitly shifted QR algorithm applied on the companion pencil. A preprocessing unitary equivalence is executed on the matrix polynomial to simultaneously bring the leading matrix coefficient and the constant matrix term to triangular form before forming the companion pencil. The resulting structure allows us to stably factor each matrix of the pencil as a product of $k$ matrices of unitary-plus-rank-one form, admitting cheap and numerically reliable storage. The problem is then solved as a product core chasing eigenvalue problem. A backward error analysis is included, implying normwise backward stability after a proper scaling. Computing the eigenvectors via reordering the Schur form is discussed as well. Numerical experiments illustrate stability and efficiency of the proposed methods.
Gaussian statistics for left and right eigenvectors of complex non-Hermitian matrices

Sofiia Dubova,Kevin Yang,Horng-Tzer Yau,Jun Yin

2024-03-29

Abstract:We consider a constant-size subset of left and right eigenvectors of an $N\times N$ i.i.d. complex non-Hermitian matrix associated with the eigenvalues with pairwise distances at least $N^{-\frac12+\epsilon}$. We show that arbitrary constant rank projections of these eigenvectors are Gaussian and jointly independent.

Probability,Mathematical Physics
Real-Time Neural Computation of the Eigenvectors of A Matrix

FL LUO,YD LI

DOI: https://doi.org/10.1109/tencon.1993.320129

1993-01-01

Abstract:This paper proposes a neural network approach for computing in real-time the eigenvectors corresponding to all the eigenvalues of a matrix. We show that the proposed neural network is guaranteed to be stable and to provide the desired eigenvectors with arbitrarily small error during an elapsed time of only a few characteristic time constants of the network.<>
Computing Eigenvectors of Normal Matrices with Simple Inverse Iteration

ZY Zhang,TW Ouyang

2003-01-01

Abstract:It is well-known that if we have an approximate eigenvalue (lambda) over cap of a normal matrix A of order n, a good approximation to the corresponding eigenvector u can be computed by one inverse iteration provided the position, say k(max), of the largest component of u is known. In this paper we give a detailed theoretical analysis to show relations between the eigenvector u and vector x(k), k = 1,..., n, obtained by simple inverse iteration, i.e., the solution to the system (A - (lambda) over capI)x = e(k) with e(k) the kth column of the identity matrix I. We prove that under some weak conditions, the index k(max) is of some optimal properties related to the smallest residual and smallest approximation error to u in spectral norm and Frobenius norm. We also prove that the normalized absolute vector v = \u\/parallel touparallel to(infinity) of u can be approximated by the normalized vector of (parallel tox(1)parallel to(2),..., parallel tox(n)parallel to(2))(T). We also give some upper bounds of \u(k)\ for those "optimal" indexes such as Fernando's heuristic for k(max) without any assumptions. A stable double orthogonal factorization method and a simpler but may less stable approach are proposed for locating the largest component of u.
Average-case analysis of the Gaussian Elimination with Partial Pivoting

Han Huang,Konstantin Tikhomirov

2024-03-06

Abstract:The Gaussian Elimination with Partial Pivoting (GEPP) is a classical algorithm for solving systems of linear equations. Although in specific cases the loss of precision in GEPP due to roundoff errors can be very significant, empirical evidence strongly suggests that for a {\it typical} square coefficient matrix, GEPP is numerically stable. We obtain a (partial) theoretical justification of this phenomenon by showing that, given the random $n\times n$ standard Gaussian coefficient matrix $A$, the {\it growth factor} of the Gaussian Elimination with Partial Pivoting is at most polynomially large in $n$ with probability close to one. This implies that with probability close to one the number of bits of precision sufficient to solve $Ax = b$ to $m$ bits of accuracy using GEPP is $m+O(\log n)$, which improves an earlier estimate $m+O(\log^2 n)$ of Sankar, and which we conjecture to be optimal by the order of magnitude. We further provide tail estimates of the growth factor which can be used to support the empirical observation that GEPP is more stable than the Gaussian Elimination with no pivoting.

Numerical Analysis,Probability
To solving the eigenvalue problem for polynomial matrices of general form

V. N. Kublanovskaya

DOI: https://doi.org/10.1007/s10958-012-0791-6

2012-04-18

Journal of Mathematical Sciences

Abstract:The paper considers the eigenvalve problem for a polynomial m × n matrix F(μ) of rank ρ. Algorithms. allowing one to reduce this problem to the generalized matrix eigenvalve problem are suggested. The algorithms are based on combining rank factorization methods with the method of hereditary pencils. Methods for exhausting the subspaces of polynomial solutions of zero index from the matrix nullspaces and for isolating the regular kernel from F(μ), with its subsequent linearization, are proposed. Bibliography: 6 titles.
Faster Algorithms for Average-Case Orthogonal Vectors and Closest Pair Problems

Josh Alman,Alexandr Andoni,Hengjie Zhang

2024-10-30

Abstract:We study the average-case version of the Orthogonal Vectors problem, in which one is given as input $n$ vectors from $\{0,1\}^d$ which are chosen randomly so that each coordinate is $1$ independently with probability $p$. Kane and Williams [ITCS 2019] showed how to solve this problem in time $O(n^{2 - \delta_p})$ for a constant $\delta_p > 0$ that depends only on $p$. However, it was previously unclear how to solve the problem faster in the hardest parameter regime where $p$ may depend on $d$. The best prior algorithm was the best worst-case algorithm by Abboud, Williams and Yu [SODA 2014], which in dimension $d = c \cdot \log n$, solves the problem in time $n^{2 - \Omega(1/\log c)}$. In this paper, we give a new algorithm which improves this to $n^{2 - \Omega(\log\log c /\log c)}$ in the average case for any parameter $p$. As in the prior work, our algorithm uses the polynomial method. We make use of a very simple polynomial over the reals, and use a new method to analyze its performance based on computing how its value degrades as the input vectors get farther from orthogonal. To demonstrate the generality of our approach, we also solve the average-case version of the closest pair problem in the same running time.

Data Structures and Algorithms
A Quantum Speed-Up for Approximating the Top Eigenvectors of a Matrix

Yanlin Chen,András Gilyén,Ronald de Wolf

2024-05-24

Abstract:Finding a good approximation of the top eigenvector of a given $d\times d$ matrix $A$ is a basic and important computational problem, with many applications. We give two different quantum algorithms that, given query access to the entries of a Hermitian matrix $A$ and assuming a constant eigenvalue gap, output a classical description of a good approximation of the top eigenvector: one algorithm with time complexity $\mathcal{\tilde{O}}(d^{1.75})$ and one with time complexity $d^{1.5+o(1)}$ (the first algorithm has a slightly better dependence on the $\ell_2$-error of the approximating vector than the second, and uses different techniques of independent interest). Both of our quantum algorithms provide a polynomial speed-up over the best-possible classical algorithm, which needs $\Omega(d^2)$ queries to entries of $A$, and hence $\Omega(d^2)$ time. We extend this to a quantum algorithm that outputs a classical description of the subspace spanned by the top-$q$ eigenvectors in time $qd^{1.5+o(1)}$. We also prove a nearly-optimal lower bound of $\tilde{\Omega}(d^{1.5})$ on the quantum query complexity of approximating the top eigenvector.

Quantum Physics,Data Structures and Algorithms
Smallest gaps between eigenvalues of real Gaussian matrices

Patrick Lopatto,Matthew Meeker

2024-03-29

Abstract:We consider an $n\times n$ matrix of independent real Gaussian random variables and determine the asymptotic distribution of the smallest gaps between complex eigenvalues.

Probability,Mathematical Physics
Exact Algorithms for Computing Generalized Eigenspaces of Matrices via Annihilating Polynomials

Shinichi Tajima,Katsuyoshi Ohara,Akira Terui

2024-06-28

Abstract:An effective exact method is proposed for computing generalized eigenspaces of a matrix of integers or rational numbers. Keys of our approach are the use of minimal annihilating polynomials and the concept of the Jourdan-Krylov basis. A new method, called Jordan-Krylov elimination, is introduced to design an algorithm for computing Jordan-Krylov basis. The resulting algorithm outputs generalized eigenspaces as a form of Jordan chains. Notably, in the output, components of generalized eigenvectors are expressed as polynomials in the associated eigenvalue as a variable.

Rings and Algebras,Symbolic Computation,Commutative Algebra
Average-Case Matrix Discrepancy: Asymptotics and Online Algorithms

Dmitriy Kunisky,Peiyuan Zhang

2024-07-20

Abstract:We study the operator norm discrepancy of i.i.d. random matrices, initiating the matrix-valued analog of a long line of work on the $\ell^{\infty}$ norm discrepancy of i.i.d. random vectors. First, using repurposed results on vector discrepancy and new first moment method calculations, we give upper and lower bounds on the discrepancy of random matrices. We treat i.i.d. matrices drawn from the Gaussian orthogonal ensemble (GOE) and low-rank Gaussian Wishart distributions. In both cases, for what turns out to be the "critical" number of $\Theta(n^2)$ matrices of dimension $n \times n$, we identify the discrepancy up to constant factors. Second, we give a new analysis of the matrix hyperbolic cosine algorithm of Zouzias (2011), a matrix version of an online vector discrepancy algorithm of Spencer (1977) studied for average-case inputs by Bansal and Spencer (2020), for the case of i.i.d. random matrix inputs. We both give a general analysis and extract concrete bounds on the discrepancy achieved by this algorithm for matrices with independent entries (including GOE matrices) and Gaussian Wishart matrices.

Data Structures and Algorithms,Discrete Mathematics,Combinatorics,Probability
Sublinear Time Eigenvalue Approximation via Random Sampling

Dexter, Gregory,Drineas, Petros

DOI: https://doi.org/10.1007/s00453-024-01208-5

IF: 0.909

2024-02-13

Algorithmica

Abstract:We study the problem of approximating the eigenspectrum of a symmetric matrix with bounded entries (i.e., ). We present a simple sublinear time algorithm that approximates all eigenvalues of up to additive error using those of a randomly sampled principal submatrix. Our result can be viewed as a concentration bound on the complete eigenspectrum of a random submatrix, significantly extending known bounds on just the singular values (the magnitudes of the eigenvalues). We give improved error bounds of and when the rows of can be sampled with probabilities proportional to their sparsities or their squared norms respectively. Here is the number of non-zero entries in and is its Frobenius norm. Even for the strictly easier problems of approximating the singular values or testing the existence of large negative eigenvalues (Bakshi, Chepurko, and Jayaram, FOCS '20), our results are the first that take advantage of non-uniform sampling to give improved error bounds. From a technical perspective, our results require several new eigenvalue concentration and perturbation bounds for matrices with bounded entries. Our non-uniform sampling bounds require a new algorithmic approach, which judiciously zeroes out entries of a randomly sampled submatrix to reduce variance, before computing the eigenvalues of that submatrix as estimates for those of . We complement our theoretical results with numerical simulations, which demonstrate the effectiveness of our algorithms in practice.

computer science, software engineering,mathematics, applied
Sublinear Time Spectral Density Estimation

Vladimir Braverman,Aditya Krishnan,Christopher Musco

DOI: https://doi.org/10.1145/3519935.3520009

2022-04-15

Abstract:We present a new sublinear time algorithm for approximating the spectral density (eigenvalue distribution) of an $n\times n$ normalized graph adjacency or Laplacian matrix. The algorithm recovers the spectrum up to $\epsilon$ accuracy in the Wasserstein-1 distance in $O(n\cdot \text{poly}(1/\epsilon))$ time given sample access to the graph. This result compliments recent work by David Cohen-Steiner, Weihao Kong, Christian Sohler, and Gregory Valiant (2018), which obtains a solution with runtime independent of $n$, but exponential in $1/\epsilon$. We conjecture that the trade-off between dimension dependence and accuracy is inherent. Our method is simple and works well experimentally. It is based on a Chebyshev polynomial moment matching method that employees randomized estimators for the matrix trace. We prove that, for any Hermitian $A$, this moment matching method returns an $\epsilon$ approximation to the spectral density using just $O({1}/{\epsilon})$ matrix-vector products with $A$. By leveraging stability properties of the Chebyshev polynomial three-term recurrence, we then prove that the method is amenable to the use of coarse approximate matrix-vector products. Our sublinear time algorithm follows from combining this result with a novel sampling algorithm for approximating matrix-vector products with a normalized graph adjacency matrix. Of independent interest, we show a similar result for the widely used \emph{kernel polynomial method} (KPM), proving that this practical algorithm nearly matches the theoretical guarantees of our moment matching method. Our analysis uses tools from Jackson's seminal work on approximation with positive polynomial kernels.

Data Structures and Algorithms,Numerical Analysis
A Proletarian Approach to Generating Eigenvalues of GUE Matrices

Luc Devroye,Jad Hamdan

2023-09-14

Abstract:We propose a simple algorithm to generate random variables described by densities equaling squared Hermite functions. Using results from random matrix theory, we utilize this to generate a randomly chosen eigenvalue of a matrix from the Gaussian Unitary Ensemble (GUE) in sublinear expected time in the RAM model.

Probability
On Eigenvector Approximation of Diagonalizable Random Matrices with Random Perturbations: Properties and Applications

Kun Chen,Zhihua Zhang

2024-10-29

Abstract:We extend the result on the top eigenvalue of the i.i.d.\ matrix with fixed perturbations by Tao to random perturbations. In particular, we consider a setup that $\mathbf{M}=\mathbf{W}+\lambda\mathbf{u}\mathbf{u}^*$ with $\mathbf{W}$ drawn from a Ginibre Orthogonal Ensemble and the perturbation $\mathbf{u}$ drawn uniformly from $\mathcal{S}^{d-1}$. We provide several asymptotic properties about the eigenvalues and the top eigenvector of the random matrix, which can not be obtained trivially from the deterministic perturbation case. We also apply our results to extend the work of Max Simchowitz, which provides an optimal lower bound for approximating the eigenspace of a symmetric matrix. We present a \textit{query complexity} lower bound for approximating the eigenvector of any asymmetric but diagonalizable matrix $\mathbf{M}$ corresponding to the largest eigenvalue. We show that for every $\operatorname{gap}\in (0,1/2]$ and large enough dimension $d$, there exists a random matrix $\mathbf{M}$ with $\operatorname{gap}(\mathbf{M})=\Omega(\operatorname{gap})$, such that if a matrix-vector query product algorithm can identity a vector $\hat{\mathbf{v}}$ which satisfies $\left\|\hat{\mathbf{v}}-\mathbf{v}_1(\mathbf{M}) \right\|_2^2\le \operatorname{const}\times \operatorname{gap}$, it needs at least $\mathcal{O}\left(\frac{\log d}{\operatorname{gap}}\right)$ queries of matrix-vector products. In the inverse polynomial accuracy regime where $\epsilon \ge \frac{1}{\operatorname{poly}(d)}$, the complexity matches the upper bounds $\mathcal{O}\left(\frac{\log(d/\epsilon)}{\operatorname{gap}}\right)$, which can be obtained via the power method. As far as we know, it is the first lower bound for computing the eigenvector of an asymmetric matrix, which is far more complicated than in the symmetric case.

Information Theory,Data Structures and Algorithms,Numerical Analysis,Statistics Theory
Central limit theorem for the complex eigenvalues of Gaussian random matrices

Advay Goel,Patrick Lopatto,Xiaoyu Xie

2024-03-09

Abstract:We establish a central limit theorem for the eigenvalue counting function of a matrix of real Gaussian random variables.

Probability,Mathematical Physics
Fast computation of spectral densities for generalized eigenvalue problems

Yuanzhe Xi,Ruipeng Li,Yousef Saad

DOI: https://doi.org/10.48550/arXiv.1706.06610

2017-06-21

Abstract:The distribution of the eigenvalues of a Hermitian matrix (or of a Hermitian matrix pencil) reveals important features of the underlying problem, whether a Hamiltonian system in physics, or a social network in behavioral sciences. However, computing all the eigenvalues explicitly is prohibitively expensive for real-world applications. This paper presents two types of methods to efficiently estimate the spectral density of a matrix pencil $(A, B)$ when both $A$ and $B$ are Hermitian and, in addition, $B$ is positive definite. The first one is based on the Kernel Polynomial Method (KPM) and the second on Gaussian quadrature by the Lanczos procedure. By employing Chebyshev polynomial approximation techniques, we can avoid direct factorizations in both methods, making the resulting algorithms suitable for large matrices. Under some assumptions, we prove bounds that suggest that the Lanczos method converges twice as fast as the KPM method. Numerical examples further indicate that the Lanczos method can provide more accurate spectral densities when the eigenvalue distribution is highly non-uniform. As an application, we show how to use the computed spectral density to partition the spectrum into intervals that contain roughly the same number of eigenvalues. This procedure, which makes it possible to compute the spectrum by parts, is a key ingredient in the new breed of eigensolvers that exploit "spectrum slicing".

Numerical Analysis
Computing Krylov iterates in the time of matrix multiplication

Vincent Neiger,Clément Pernet,Gilles Villard

DOI: https://doi.org/10.1145/3666000.3669715

2024-08-20

Abstract:Krylov methods rely on iterated matrix-vector products $A^k u_j$ for an $n\times n$ matrix $A$ and vectors $u_1,\ldots,u_m$. The space spanned by all iterates $A^k u_j$ admits a particular basis -- the \emph{maximal Krylov basis} -- which consists of iterates of the first vector $u_1, Au_1, A^2u_1,\ldots$, until reaching linear dependency, then iterating similarly the subsequent vectors until a basis is obtained. Finding minimal polynomials and Frobenius normal forms is closely related to computing maximal Krylov bases. The fastest way to produce these bases was, until this paper, Keller-Gehrig's 1985 algorithm whose complexity bound $O(n^\omega \log(n))$ comes from repeated squarings of $A$ and logarithmically many Gaussian eliminations. Here $\omega>2$ is a feasible exponent for matrix multiplication over the base field. We present an algorithm computing the maximal Krylov basis in $O(n^\omega\log\log(n))$ field operations when $m \in O(n)$, and even $O(n^\omega)$ as soon as $m\in O(n/\log(n)^c)$ for some fixed real $c>0$. As a consequence, we show that the Frobenius normal form together with a transformation matrix can be computed deterministically in $O(n^\omega (\log\log(n))^2)$, and therefore matrix exponentiation~$A^k$ can be performed in the latter complexity if $\log(k) \in O(n^{\omega-1-\varepsilon})$ for some fixed $\varepsilon>0$. A key idea for these improvements is to rely on fast algorithms for $m\times m$ polynomial matrices of average degree $n/m$, involving high-order lifting and minimal kernel bases.

Symbolic Computation
On the worst-case complexity of integer Gaussian elimination

Xin Gui Fang,George Havas

DOI: https://doi.org/10.1145/258726.258740

1997-01-01

Abstract:Gaussian elimination is the basis for classical algorithms for computing canonical forms of integer matrices. Experimental results have shown that integer Gaussian elimination may lead to rapid growth of intermediate entries. On the other hand various polynomial time algorithms do exist for such computations, but these algorithms are relatively complicated to describe and understand. Gaussian elimination provides the simplest descriptions of algorithms for this purpose. These algorithms have a nice polynomial number of steps, but the steps deal with long operands. Here we show that there is an exponential length lower bound on the operands for a well-deflned variant of Gaussian elimination when applied to Smith and Hermite normal form calculation. We present explicit matrices for which this variant produces exponential length entries. Thus, Gaussian elimination has worst-case exponential space and time complexity for such applications. The analysis provides guidance as to how integer matrix algorithms based on Gaussian elimination may be further developed for better performance, which is important since many practical algorithms for computing canonical forms are so based.

Average polynomial time for eigenvector computations

Fast and backward stable computation of the eigenvalues and eigenvectors of matrix polynomials

Gaussian statistics for left and right eigenvectors of complex non-Hermitian matrices

Real-Time Neural Computation of the Eigenvectors of A Matrix

Computing Eigenvectors of Normal Matrices with Simple Inverse Iteration

Average-case analysis of the Gaussian Elimination with Partial Pivoting

To solving the eigenvalue problem for polynomial matrices of general form

Faster Algorithms for Average-Case Orthogonal Vectors and Closest Pair Problems

A Quantum Speed-Up for Approximating the Top Eigenvectors of a Matrix

Smallest gaps between eigenvalues of real Gaussian matrices

Exact Algorithms for Computing Generalized Eigenspaces of Matrices via Annihilating Polynomials

Average-Case Matrix Discrepancy: Asymptotics and Online Algorithms

Sublinear Time Eigenvalue Approximation via Random Sampling

Sublinear Time Spectral Density Estimation

A Proletarian Approach to Generating Eigenvalues of GUE Matrices

On Eigenvector Approximation of Diagonalizable Random Matrices with Random Perturbations: Properties and Applications

Central limit theorem for the complex eigenvalues of Gaussian random matrices

Fast computation of spectral densities for generalized eigenvalue problems

Computing Krylov iterates in the time of matrix multiplication

On the worst-case complexity of integer Gaussian elimination