Abstract:This paper presents two novel algorithms for approximately projecting symmetric matrices onto the Positive Semidefinite (PSD) cone using Randomized Numerical Linear Algebra (RNLA). Classical PSD projection methods rely on full-rank deterministic eigen-decomposition, which can be computationally prohibitive for large-scale problems. Our approach leverages RNLA to construct low-rank matrix approximations before projection, significantly reducing the required numerical resources. The first algorithm utilizes random sampling to generate a low-rank approximation, followed by a standard eigen-decomposition on this smaller matrix. The second algorithm enhances this process by introducing a scaling approach that aligns the leading-order singular values with the positive eigenvalues, ensuring that the low-rank approximation captures the essential information about the positive eigenvalues for PSD projection. Both methods offer a trade-off between accuracy and computational speed, supported by probabilistic error bounds. To further demonstrate the practical benefits of our approach, we integrate the randomized projection methods into a first-order Semi-Definite Programming (SDP) solver. Numerical experiments, including those on SDPs derived from Sum-of-Squares (SOS) programming problems, validate the effectiveness of our method, especially for problems that are infeasible with traditional deterministic methods.
What problem does this paper attempt to address?
### What problem does this paper attempt to solve?
This paper aims to solve the problem of projecting large - scale symmetric matrices onto the positive semidefinite (PSD) cone. Specifically, traditional PSD projection methods rely on full - rank deterministic eigenvalue decomposition, and this method has a prohibitively high computational cost when dealing with large - scale problems and is difficult to implement. Therefore, the authors propose two new algorithms based on randomized numerical linear algebra (RNLA) to approximately project symmetric matrices onto the PSD cone, thereby significantly reducing the required numerical resources.
### Specific problem description
1. **High computational complexity**: The time complexity of traditional PSD projection methods is \(O(n^3)\). For large - scale matrices (such as when \(n\) is large), this computational complexity makes the method impractical.
2. **Feasibility of large - scale problems**: Directly solving optimization problems involving PSD constraints (such as semidefinite programming, SDP) is also infeasible in large - scale cases because although these constraints are convex, they are costly to encode.
3. **Limitations of existing methods**: Existing approximate PSD projection methods still require \(O(n^3)\) floating - point operations (FLOPs), which is still a bottleneck when \(n\) is very large.
### Solutions
The solution proposed by the authors is to use RNLA techniques to construct low - rank matrix approximations and perform PSD projections on this basis. Specifically, it includes the following two algorithms:
1. **The first algorithm**: Generate a low - rank approximation through random sampling, and then perform a standard eigenvalue decomposition on this small matrix.
2. **The second algorithm**: Introduce a scaling method to align the main singular values with the positive eigenvalues, ensuring that the low - rank approximation can capture key information about the positive eigenvalues, thereby better completing the PSD projection.
These two methods provide a trade - off between accuracy and computational speed and have probabilistic error - bound guarantees. In addition, the authors also integrate these randomized projection methods into a first - order semidefinite programming (SDP) solver and verify their effectiveness through numerical experiments, especially in large - scale problems that cannot be handled by traditional deterministic methods.
### Mathematical formula representation
- The PSD projection problem can be formalized as a convex optimization problem:
\[
X^+=\arg\min_{Y\succeq0}\|X - Y\|_F^2
\]
where \(X\in S_n\) is a symmetric matrix, and \(S_n^+\) represents the set of symmetric positive semidefinite matrices.
- The closed - form solution of traditional PSD projection:
\[
X^+ = UD^+U^\top
\]
where \(X = UDU^\top\) is the eigenvalue decomposition of \(X\), and \(D^+ \) is the result of setting the negative eigenvalues in \(D\) to zero.
- The core idea of the randomized PSD projection algorithm is to use the low - rank approximation \(QQ^\top X QQ^\top\), where \(Q\in\mathbb{R}^{n\times k}\) is a matrix obtained through random sampling and QR decomposition, satisfying \(Q^\top Q = I\).
Through these methods, the authors show how to effectively perform PSD projection in large - scale problems, thereby solving the computational inefficiency of traditional methods.