Christos Boutsikas,Petros Drineas,Ilse C.F. Ipsen
Abstract:We perturb a real matrix $A$ of full column rank, and derive lower bounds for the smallest singular values of the perturbed matrix, in terms of normwise absolute perturbations. Our bounds, which extend existing lower-order expressions, demonstrate the potential increase in the smallest singular values, and represent a qualitative model for the increase in the small singular values after a matrix has been downcast to a lower arithmetic precision. Numerical experiments confirm the qualitative validity of this model and its ability to predict singular values changes in the presence of decreased arithmetic precision.
What problem does this paper attempt to address?
### Problems the paper attempts to solve
This paper explores how the minimum singular value of a real matrix \(A\) changes when it is represented with reduced precision. Specifically, the author studies how the minimum singular value of matrix \(A\) increases under low precision and provides theoretical lower bounds for these changes.
### Main contributions
1. **Theoretical lower bounds**: The author derives the lower bounds of the minimum singular value of the reduced - precision matrix \(A + E\). These lower bounds indicate that the minimum singular value may increase significantly after the reduction in precision.
2. **Numerical experiments**: The validity of these theoretical lower bounds is verified through numerical experiments, proving that in low - precision calculations, the minimum singular value does increase.
3. **Model establishment**: A qualitative model is proposed to describe the increase in the minimum singular value of a matrix after the reduction in precision. This model helps in understanding the impact of computational precision on the singular values of a matrix.
### Specific problem description
Given a real full - column - rank matrix \(A\), how will its minimum singular value change when \(A\) is represented with reduced precision? Specifically, the author focuses on:
- **Matrix precision reduction**: Reducing the matrix \(A\) from double - precision to single - precision or half - precision.
- **Singular value change**: Studying how the minimum singular value of the matrix changes after the reduction in precision.
- **Theoretical analysis**: Deriving the theoretical lower bounds of the change in the minimum singular value and verifying the accuracy of these lower bounds through numerical experiments.
### Mathematical background
- **Singular value decomposition**: Matrix \(A\) can be decomposed as \(A = U\Sigma V^{T}\), where \(U\) and \(V\) are orthogonal matrices and \(\Sigma\) is a diagonal matrix, and the elements on the diagonal are the singular values of \(A\).
- **Reduced - precision representation**: Reduced - precision representation will cause a small perturbation \(E\) in matrix \(A\), thus forming a new matrix \(A + E\).
- **Singular value change**: According to Weyl's inequality, the range of change of the singular values of matrix \(A\) under perturbation \(E\) can be expressed as:
\[
|\sigma_j(A + E)-\sigma_j(A)|\leq\|E\|_2
\]
### Theoretical results
- **Single minimum singular value**: For a matrix with only one minimum singular value, the author derives its lower bound after perturbation.
- **Multiple minimum singular values**: For a matrix with a cluster of minimum singular values, the author also derives the lower bounds of these singular values after perturbation.
### Numerical experiments
- **Generating matrices**: Use the Julia programming language to generate a diagonal matrix \(\Sigma\) containing exact singular values and generate matrix \(A\).
- **Calculating singular values**: Calculate the singular values of matrix \(A\) and its low - precision versions to verify the accuracy of the theoretical lower bounds.
### Conclusion
Through theoretical analysis and numerical experiments, the author has proven that in low - precision calculations, the minimum singular value of a matrix does increase. This finding is of great significance for understanding the impact of computational precision on the singular values of a matrix.