Abstract:Recent development on mixed precision techniques has largely enhanced the performance of various linear algebra solvers, one of which being the least squares problem $\min_{x}\lVert b-Ax\rVert_{2}$. By transforming the least squares problem into an augmented linear system, mixed precision techniques are capable of refining the lower precision solution to the working precision. In this paper, we propose mixed precision iterative refinement algorithms for two variants of the least squares problem -- the least squares problem with linear equality constraints (LSE) and the generalized least squares problem (GLS). Both classical and GMRES-based iterative refinement can be applied to augmented systems of these two problems to improve the accuracy of the solution. For reasonably well-conditioned problems our algorithms reduce the execution time by a factor of 40% in average compared to the fixed precision ones from LAPACK on the x86-64 architecture.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: how to efficiently and accurately solve the least squares problem with linear equality constraints (LSE, Least Squares with Linear Equality Constraints) and the generalized least squares problem (GLS, Generalized Least Squares) through mixed - precision iterative refinement algorithms. Specifically, the paper proposes the classical iterative refinement algorithms and GMRES - based iterative refinement algorithms for these two least - squares problems, and uses the generalized RQ or QR decomposition for the initial solution. Through these methods, the computational time can be significantly reduced while ensuring the accuracy of the solution.
### Detailed Explanation:
1. **Problem Background**:
- The least - squares problem is very important in scientific computing and numerical linear algebra, and is widely used in physics, engineering, computer vision and other fields.
- The least - squares problem with linear equality constraints (LSE) and the generalized least - squares problem (GLS) are two important types of least - squares problems.
2. **LSE Problem Description**:
Given matrices \( A\in\mathbb{C}^{m\times n}\) and \( B\in\mathbb{C}^{p\times n}\), where \( p\leq n\leq m + p\), the LSE problem can be expressed as:
\[
\min_{x\in\mathbb{C}^n}\|Ax - b\|_2\quad\text{s.t.}\quad Bx = d.
\]
If the following conditions are met:
\[
\text{rank}(B)=p,\quad\text{rank}\left(\begin{bmatrix}A\\B\end{bmatrix}\right)=n,
\]
then the system has a unique solution. The LSE problem is widely used in application regression analysis, constrained surface fitting, constrained optimization and other fields.
3. **GLS Problem Description**:
Given matrices \( W\in\mathbb{C}^{n\times m}\) and \( V\in\mathbb{C}^{n\times p}\), where \( m\leq n\leq m + p\), the GLS problem can be expressed as:
\[
\min_{x\in\mathbb{C}^m,y\in\mathbb{C}^p}\|y\|_2\quad\text{s.t.}\quad Wx+Vy = d.
\]
If the following conditions are met:
\[
\text{rank}(W)=m,\quad\text{rank}\left([W,V]\right)=n,
\]
then the system is always consistent and there exist a unique solution \( x\) and a minimum 2 - norm solution \( y\). The GLS problem has important applications in economics and engineering, for example, for the least - squares estimation of the linear model \( d = Wx+\epsilon\), where \(\epsilon\) is a zero - mean random noise vector.
4. **Advantages of Mixed - Precision Algorithms**:
- Mixed - precision algorithms accelerate the calculation by using lower - precision arithmetic operations (such as half - precision fp16), and at the same time improve the solution to the working precision through the iterative refinement method, thereby significantly increasing the calculation speed without sacrificing accuracy.
- The methods proposed in the paper can significantly improve the performance on modern accelerators (such as GPUs), especially for problems with a moderate condition number, and can reduce the execution time by an average of 40%.
5. **Main Contributions**:
- Proposed mixed - precision classical iterative refinement algorithms and GMRES - based iterative refinement algorithms applicable to LSE and GLS problems.