High-precision randomized iterative methods for the random feature method

Jingrun Chen,Longze Tan
2024-09-24
Abstract:This paper focuses on solving large-scale, ill-conditioned, and overdetermined sparse least squares problems that arise from numerical partial differential equations (PDEs), mainly from the random feature method. To address these difficulties, we introduce (1) a count sketch technique to sketch the original matrix to a smaller matrix; (2) a QR factorization or a singular value decomposition for the smaller matrix to obtain the preconditioner, which is multiplied to the original matrix from the right-hand side; (3) least squares iterative solvers to solve the preconditioned least squares system. Therefore, the methods we develop are termed CSQRP-LSQR and CSSVDP-LSQR. Under mild assumptions, we prove that the preconditioned problem holds a condition number whose upper bound is independent of the condition number of the original matrix, and provide error estimates for both methods. Ample numerical experiments, including least squares problems arising from two-dimensional and three-dimensional PDEs and the Florida Sparse Matrix Collection, are conducted. Both methods are comparable to or even better than direct methods in accuracy and are computationally more efficient for large-scale problems. This opens up the applicability of the random feature method for PDEs over complicated geometries with high-complexity solutions.
Numerical Analysis
What problem does this paper attempt to address?
This paper aims to solve large - scale, ill - conditioned and over - determined sparse least - squares problems arising from numerical partial differential equations (PDEs). Specifically, these problems mainly stem from the Random Feature Method (RFM). To address these difficulties, the authors propose the following methods: 1. **Count Sketch Technique**: Compress the original matrix into a smaller matrix. 2. **QR decomposition or Singular Value Decomposition (SVD)**: Perform QR decomposition or SVD on the smaller matrix to obtain a pre - processing matrix and multiply it on the right side of the original matrix. 3. **Least - squares iterative solver**: Used to solve the pre - processed least - squares system. Therefore, the developed methods are named CSQRP - LSQR and CSSVDP - LSQR. Under mild assumptions, the authors prove that the upper bound of the condition number of the pre - processed problem is independent of the condition number of the original matrix and provide error estimates for the two methods. Through a large number of numerical experiments, including 2D and 3D PDEs and problems in the Florida Sparse Matrix Collection, these two methods are comparable or even better in accuracy to direct methods and are more computationally efficient for large - scale problems. This opens up new possibilities for the application of the random feature method in solutions of high - complexity PDEs on complex geometries. ### Keywords - Random Feature Method - Least - squares problems - Ill - conditioned - Count Sketch matrix - Random iterative method ### Mathematical problem description The least - squares problem can be expressed as: \[ \min_{x \in \mathbb{R}^n} \|Ax - b\|_2^2, \] where \( A \in \mathbb{R}^{m \times n} \) and \( b \in \mathbb{R}^m \). The geometric complexity in practical applications leads to \( m \gg n \), and usually \( m \) is several orders of magnitude larger than \( n \). If \( A \) has full column rank, then the problem (1.1) has a unique solution, which can be obtained by direct methods such as Singular Value Decomposition (SVD) and QR decomposition. The computational complexity is \( O(mn \min(m, n)) \). While the computational complexity of each step of the iterative method is \( O(mn) \), and the total cost depends on the number of iterations required to reach a given tolerance. ### Method overview 1. **Count Sketch Technique**: Compress the original matrix \( A \) into a smaller matrix \( \tilde{A} \). 2. **QR decomposition or SVD**: Perform QR decomposition or SVD on the smaller matrix \( \tilde{A} \) to obtain a pre - processing matrix \( R \) or \( P \). 3. **Least - squares iterative solver**: Use the LSQR method to solve the pre - processed least - squares system \( \min_{y \in \mathbb{R}^n} \|By - b\|_2 \), where \( B = AR^{-1} \) or \( B = AP \). ### Experimental results Through a large number of numerical experiments, including 2D and 3D PDEs and problems in the Florida Sparse Matrix Collection, the CSQRP - LSQR and CSSVDP - LSQR methods are comparable or even better in accuracy to direct methods and are more computationally efficient for large - scale problems. This opens up new possibilities for the application of the random feature method in solutions of high - complexity PDEs on complex geometries. ### Conclusion The CSQRP - LSQR and CSSVDP - LSQR methods proposed in this paper perform well in solving large - scale, ill - conditioned and over - determined sparse least - squares problems, especially when dealing with high - complexity PDEs on complex geometries. These methods are not only comparable or even better in accuracy to direct methods, but also have significant advantages in computational efficiency.