Ethan N. Epperly,Joel A. Tropp,Robert J. Webber
Abstract:The implicit trace estimation problem asks for an approximation of the trace of a square matrix, accessed via matrix-vector products (matvecs). This paper designs new randomized algorithms, XTrace and XNysTrace, for the trace estimation problem by exploiting both variance reduction and the exchangeability principle. For a fixed budget of matvecs, numerical experiments show that the new methods can achieve errors that are orders of magnitude smaller than existing algorithms, such as the Girard-Hutchinson estimator or the Hutch++ estimator. A theoretical analysis confirms the benefits by offering a precise description of the performance of these algorithms as a function of the spectrum of the input matrix. The paper also develops an exchangeable estimator, XDiag, for approximating the diagonal of a square matrix using matvecs.
What problem does this paper attempt to address?
The core problem that this paper attempts to solve is the **implicit matrix trace estimation problem**, that is, to approximately calculate the trace of a square matrix through matrix - vector multiplications (matvecs). Specifically, given a square matrix \(A\) that can only be accessed through matrix - vector multiplications, how to efficiently and accurately estimate its trace \(\text{tr}(A)\).
### Main contributions of the paper
1. **New algorithm design**:
- Two new randomized algorithms, **XTrace** and **XNysTrace**, are proposed.
- These algorithms utilize **variance reduction** and the **exchangeability principle**, and thus can significantly reduce the estimation error under the same matrix - vector multiplication budget.
2. **Theoretical analysis**:
- Through theoretical analysis, the performance advantages of these new algorithms are proved. In particular, their variances decrease at a rate of \(O(1/m^2)\) as the number of matrix - vector multiplications \(m\) increases, which is better than the convergence rate of \(O(1/m)\) of the traditional Monte Carlo method.
- For matrices with rapidly decaying singular values, the convergence speeds of XTrace and XNysTrace are 1.5 times and 3 times faster than that of the existing method Hutch++, respectively.
3. **Numerical experiments**:
- Through a series of numerical experiments, the effectiveness of the new algorithms is verified. In particular, when dealing with matrices with exponentially decaying eigenvalues, the errors of XTrace and XNysTrace can be several orders of magnitude lower than those of the existing methods.
- In practical applications, such as calculating the partition function of a quantum system, XTrace and XNysTrace also show significant advantages.
### Explanation of key concepts
- **Exchangeability principle**: If the test vectors \(\omega_1,\ldots,\omega_k\) are exchangeable (that is, their distributions do not depend on the order), then the minimum - variance unbiased estimator must be a symmetric function of these vectors.
- **Variance reduction**: Reduce the variance of the estimator by using techniques such as low - rank approximation, thereby improving the estimation accuracy.
- **Matrix trace estimation**: Calculate the trace of a matrix (that is, the sum of its diagonal elements), but the matrix can only be accessed through matrix - vector multiplications.
### Summary
By introducing the exchangeability principle and variance reduction techniques, the paper designs new randomized algorithms XTrace and XNysTrace for the implicit matrix trace estimation problem. These new algorithms not only have better performance guarantees in theory, but also show significant advantages in practical applications.