A simple way to speedup Gauss Elimination

Sivaram Ambikasaran

DOI: https://doi.org/10.48550/arXiv.1101.4081

2011-01-21

Numerical Analysis

Abstract:This article looks at a simple modification to speedup the conventional Gauss Elimination. The proposed modification speeds up the conventional Gauss Elimination by a factor of nearly 9/7 (in the asymptotic limit).

What problem does this paper attempt to address?

The Modified Solutoin for Large Sparse Symmetric Linear Systems in Electromagnetic Field Analysis

Jinming Wang,Dexin Xie,Yingying Yao

DOI: https://doi.org/10.3321/j.issn:1000-6753.2001.02.007

2001-01-01

Abstract:A modified ICCG method for large sparse symmetric linear equations in finite element analysis of electromagnetic field problems is presented in the paper. A controlling parameter is used to reduce the amount of the non-zero elements in the process of the incomplete Cholesky decomposition, so as to reduce the computation time of the incomplete decomposition and each iterative step of the conjugate gradient method. Through theoretical analysis, the proper choice of the controlling parameter not only has no influence on the rate of convergence, but also accelerates the convergence sometimes. Numerical examples show that the method can reduce the computation time by 30 ％～50％ than the ICCG (or PCBCG) method.
Approximate Gaussian Elimination for Laplacians: Fast, Sparse, and Simple

Rasmus Kyng,Sushant Sachdeva

DOI: https://doi.org/10.48550/arXiv.1605.02353

2016-05-09

Abstract:We show how to perform sparse approximate Gaussian elimination for Laplacian matrices. We present a simple, nearly linear time algorithm that approximates a Laplacian by a matrix with a sparse Cholesky factorization, the version of Gaussian elimination for symmetric matrices. This is the first nearly linear time solver for Laplacian systems that is based purely on random sampling, and does not use any graph theoretic constructions such as low-stretch trees, sparsifiers, or expanders. The crux of our analysis is a novel concentration bound for matrix martingales where the differences are sums of conditionally independent variables.

Data Structures and Algorithms
Average-case analysis of the Gaussian Elimination with Partial Pivoting

Han Huang,Konstantin Tikhomirov

2024-03-06

Abstract:The Gaussian Elimination with Partial Pivoting (GEPP) is a classical algorithm for solving systems of linear equations. Although in specific cases the loss of precision in GEPP due to roundoff errors can be very significant, empirical evidence strongly suggests that for a {\it typical} square coefficient matrix, GEPP is numerically stable. We obtain a (partial) theoretical justification of this phenomenon by showing that, given the random $n\times n$ standard Gaussian coefficient matrix $A$, the {\it growth factor} of the Gaussian Elimination with Partial Pivoting is at most polynomially large in $n$ with probability close to one. This implies that with probability close to one the number of bits of precision sufficient to solve $Ax = b$ to $m$ bits of accuracy using GEPP is $m+O(\log n)$, which improves an earlier estimate $m+O(\log^2 n)$ of Sankar, and which we conjecture to be optimal by the order of magnitude. We further provide tail estimates of the growth factor which can be used to support the empirical observation that GEPP is more stable than the Gaussian Elimination with no pivoting.

Numerical Analysis,Probability
Cuda Based Parallel Computation For Gauss Elimination Method

Xiao Liu,Lei Xu

DOI: https://doi.org/10.1115/omae2018-78479

2018-01-01

Abstract:The Central Processing Unit (CPU) parallel algorithm based on Computing Unified Device Architecture (CUDA) has shown great power of computing speedup ability. What performance will the new technique show in the field of structural computation? We choose the Gauss elimination method as the research object. In this study, the parallel Gauss elimination is realized in CUDA on GPU. Furthermore, we carry out two groups of numerical experiments. The first group investigates the effect of Matrix Bandwidths (MBs) and Node Numbers (NNs) on speedup ratio. The second one compares our method with the commercial software by analyzing two actual structural problems in ocean engineering.
Generalized elimination method and its applications to matrix factorizations

E. Golpar-Raboky

DOI: https://doi.org/10.1007/s13226-024-00678-1

2024-09-20

Indian Journal of Pure and Applied Mathematics

Abstract:An elimination method is a sequence of elementary row or column operations to divide a matrix into parts with partial zeroing of columns or rows of the matrix leading to a matrix factorization implicitly. We present a generalized elimination algorithm for solving real and Diophantine linear systems. The algorithm uses a zeroing process based on two index sets and provides a mechanism for constructing factorizations such as block LU factorization as well as the WZ and the ZW factorizations under a common framework. We also introduce the octant interlocking factorization method and present two new factorizations named as the ODS and the SDO factorizations. Necessary and sufficient conditions under which the generalized elimination method is carried out are proposed.

mathematics
The asymptotic complexity of matrix reduction over finite fields

Demetres Christofides

DOI: https://doi.org/10.48550/arXiv.1406.5826

2014-06-23

Data Structures and Algorithms

Abstract:Consider an invertible n \times n matrix over some field. The Gauss-Jordan elimination reduces this matrix to the identity matrix using at most n^2 row operations and in general that many operations might be needed. In [1] the authors considered matrices in GL(n;q), the set of n \times n invertible matrices in the finite field of q elements, and provided an algorithm using only row operations which performs asymptotically better than the Gauss-Jordan elimination. More specifically their `striped elimination algorithm' has asymptotic complexity \frac{n^2}{\log_q{n}}. Furthermore they proved that up to a constant factor this algorithm is best possible as almost all matrices in GL(n;g) need asymptotically at least \frac{n^2}{2\log_q{n}} operations. In this short note we show that the `striped elimination algorithm' is asymptotically optimal by proving that almost all matrices in GL(n;q) need asymptotically at least frac{n^2}{\log_q{n}} operations.
On the worst-case complexity of integer Gaussian elimination

Xin Gui Fang,George Havas

DOI: https://doi.org/10.1145/258726.258740

1997-01-01

Abstract:Gaussian elimination is the basis for classical algorithms for computing canonical forms of integer matrices. Experimental results have shown that integer Gaussian elimination may lead to rapid growth of intermediate entries. On the other hand various polynomial time algorithms do exist for such computations, but these algorithms are relatively complicated to describe and understand. Gaussian elimination provides the simplest descriptions of algorithms for this purpose. These algorithms have a nice polynomial number of steps, but the steps deal with long operands. Here we show that there is an exponential length lower bound on the operands for a well-deflned variant of Gaussian elimination when applied to Smith and Hermite normal form calculation. We present explicit matrices for which this variant produces exponential length entries. Thus, Gaussian elimination has worst-case exponential space and time complexity for such applications. The analysis provides guidance as to how integer matrix algorithms based on Gaussian elimination may be further developed for better performance, which is important since many practical algorithms for computing canonical forms are so based.
Alternating iterative method based on modified Gauss-Seidel method

Sheng Zhou,Tingzhu Huang

2006-01-01

Journal of Information and Computational Science

Abstract:Kohno et al. [5] has reported the improving modified Gauss-Seidel method by using a preconditioned matrix. In this paper we will study the convergence of an alternating iterative MGS method. A numerical example will be given.
Gaussian elimination corrects pivoting mistakes

Alex Townsend

DOI: https://doi.org/10.48550/arXiv.1602.06602

2016-02-21

Numerical Analysis

Abstract:Gaussian elimination (GE) is the archetypal direct algorithm for solving linear systems of equations and this has been its primary application for thousands of years. In the last decade, GE has found another major use as an iterative algorithm for low rank approximation. In this setting, GE is often employed with complete pivoting and designed to allow for non-optimal pivoting, i.e., pivoting mistakes, that could render GE numerically unstable when implemented in floating point arithmetic. While it may appear that pivoting mistakes could accumulate and lead to a large growth factor, we show that later GE steps correct earlier pivoting mistakes, even while more are being made. In short, GE is very robust to non-optimal pivots, allowing for its iterative variant to flourish.
Speeding Up the GVW Algorithm Via a Substituting Method

Ting Li,Yao Sun,Zhenyu Huang,Dingkang Wang,Dongdai Lin

DOI: https://doi.org/10.1007/s11424-019-8345-3

2019-01-01

Abstract:The GVW algorithm is an efficient signature-based algorithm for computing Gröbner bases. In this paper, the authors consider the implementation of the GVW algorithm by using linear algebra, and speed up GVW via a substituting method. As it is well known that, most of the computing time of a Gröbner basis is spent on reductions of polynomials. Thus, linear algebraic techniques, such as matrix operations, have been used extensively to speed up the implementations. Particularly, one-direction (also called signature-safe) reduction is used in signature-based algorithms, because polynomials (or rows in matrices) with larger signatures can only be reduced by polynomials (rows) with smaller signatures. The authors propose a new method to construct sparser matrices for signature-based algorithms via a substituting method. Specifically, instead of only storing the original polynomials in GVW, the authors also record many equivalent but sparser polynomials at the same time. In matrix construction, denser polynomials are substituted by sparser equivalent ones. As the matrices get sparser, they can be eliminated more efficiently. Two specifical algorithms, Block-GVW and LMGVW, are presented, and their combination is the Sub-GVW algorithm. The correctness of the new proposed method is proved, and the experimental results demonstrate the efficiency of this new method.
Iterative methods for solving factorized linear systems

Anna Ma,Deanna Needell,Aaditya Ramdas

DOI: https://doi.org/10.48550/arXiv.1701.07453

2019-01-10

Abstract:Stochastic iterative algorithms such as the Kaczmarz and Gauss-Seidel methods have gained recent attention because of their speed, simplicity, and the ability to approximately solve large-scale linear systems of equations without needing to access the entire matrix. In this work, we consider the setting where we wish to solve a linear system in a large matrix X that is stored in a factorized form, X = UV; this setting either arises naturally in many applications or may be imposed when working with large low-rank datasets for reasons of space required for storage. We propose a variant of the randomized Kaczmarz method for such systems that takes advantage of the factored form, and avoids computing X. We prove an exponential convergence rate and supplement our theoretical guarantees with experimental evidence demonstrating that the factored variant yields significant acceleration in convergence.

Numerical Analysis
A modification of the Jacobi-Davidson method

Mashetti Ravibabu

DOI: https://doi.org/10.48550/arXiv.1902.02285

2019-02-07

Abstract:Each iteration in Jacobi-Davidson method for solving large sparse eigenvalue problems involves two phases, called subspace expansion and eigen pair extraction. The subspace expansion phase involves solving a correction equation. We propose a modification to this by introducing a related correction equation, motivated by the least squares. We call the proposed method as the Modified Jacobi-Davidson Method. When the subspace expansion is ignored as in the Simplified Jacobi- Davidson Method, the modified method is called as Modified Simplified Jacobi-Davidson Method. We analyze the convergence properties of the proposed method for Symmetric matrices. Numerical experiments have been carried out to check whether the method is computationally viable or not.

Numerical Analysis
Efficient Accelerated Coordinate Descent Methods and Faster Algorithms for Solving Linear Systems

Yin Tat Lee,Aaron Sidford

DOI: https://doi.org/10.1109/focs.2013.24

2013-10-01

Abstract:In this paper we show how to accelerate randomized coordinate descent methods and achieve faster convergence rates without paying per-iteration costs in asymptotic running time. In particular, we show how to generalize and efficiently implement a method proposed by Nesterov, giving faster asymptotic running times for various algorithms that use standard coordinate descent as a black box. In addition to providing a proof of convergence for this new general method, we show that it is numerically stable, efficiently implementable, and in certain regimes, asymptotically optimal. To highlight the power of this algorithm, we show how it can used to create faster linear system solvers in several regimes: • We show how this method achieves a faster asymptotic runtime than conjugate gradient for solving a broad class of symmetric positive definite systems of equations. • We improve the convergence guarantees for Kaczmarz methods, a popular technique for image reconstruction and solving overdetermined systems of equations, by accelerating an algorithm of Strohmer and Vershynin. • We achieve the best known running time for solving Symmetric Diagonally Dominant (SDD) system of equations in the unit-cost RAM model, obtaining a running time of ${\bf O}({\bf m}\log^{\wedge}\{3/2\}{\bf n}(\log\log {\bf n})^{\wedge}\{1/2\}\log((\log {\bf n})/{\bf eps}))$ by accelerating a recent solver by Kelner et al. We show how this method achieves a faster asymptotic runtime than conjugate gradient for solving a broad class of symmetric positive definite systems of equations. We improve the convergence guarantees for Kaczmarz methods, a popular technique for image reconstruction and solving overdetermined systems of equations, by accelerating an algorithm of Strohmer and Vershynin. We achieve the best known running time for solving Symmetric Diagonally Dominant (SDD) system of equations in the unit-cost RAM model, obtaining a running time of ${\bf O}({\bf m}\log^{\wedge}\{3/2\}{\bf n}(\log\log {\bf n})^{\wedge}\{1/2\}\log((\log {\bf n})/{\bf eps}))$ by accelerating a recent solver by Kelner et al. Beyond the independent interest of these solvers, we believe they highlight the versatility of the approach of this paper and we hope that they will open the door for further algorithmic improvements in the future.
An Efficient Gauss--Newton Algorithm for Symmetric Low-Rank Product Matrix Approximations

Xin Liu,Zaiwen Wen,Yin Zhang

DOI: https://doi.org/10.1137/140971464

IF: 2.763

2015-01-01

SIAM Journal on Optimization

Abstract:We derive and study a Gauss-Newton method for computing a symmetric low-rank product XXT, where X is an element of R-nxk for k < n, that is the closest to a given symmetric matrix A is an element of R-nxn in Frobenius norm. When A = (BB)-B-T (or BBT), this problem essentially reduces to finding a truncated singular value decomposition of B. Our Gauss-Newton method, which has a particularly simple form, shares the same order of iteration-complexity as a gradient method when k << n, but can be significantly faster on a wide range of problems. In this paper, we prove global convergence and a Q-linear convergence rate for this algorithm and perform numerical experiments on various test problems, including those from recently active areas of matrix completion and robust principal component analysis. Numerical results show that the proposed algorithm is capable of providing considerable speed advantages over Krylov subspace methods on suitable application problems where high-accuracy solutions are not required. Moreover, the algorithm possesses a higher degree of concurrency than Krylov subspace methods, thus offering better scalability on modern multi-/many-core computers.
Good and Fast Row-Sparse ah-Symmetric Reflexive Generalized Inverses

Gabriel Ponte,Marcia Fampa,Jon Lee,Luze Xu

2024-06-26

Abstract:We present several algorithms aimed at constructing sparse and structured sparse (row-sparse) generalized inverses, with application to the efficient computation of least-squares solutions, for inconsistent systems of linear equations, in the setting of multiple right-hand sides and a rank-deficient constraint matrix. Leveraging our earlier formulations to minimize the 1- and 2,1- norms of generalized inverses that satisfy important properties of the Moore-Penrose pseudoinverse, we develop efficient and scalable ADMM algorithms to address these norm-minimization problems and to limit the number of nonzero rows in the solution. We establish a 2,1-norm approximation result for a local-search procedure that was originally designed for 1-norm minimization, and we compare the ADMM algorithms with the local-search procedure and with general-purpose optimization solvers.

Optimization and Control
A New Upper Bound For the Growth Factor in Gaussian Elimination with Complete Pivoting

Ankit Bisain,Alan Edelman,John Urschel

2024-10-14

Abstract:The growth factor in Gaussian elimination measures how large the entries of an LU factorization can be relative to the entries of the original matrix. It is a key parameter in error estimates, and one of the most fundamental topics in numerical analysis. We produce an upper bound of $n^{0.2079 \ln n +0.91}$ for the growth factor in Gaussian elimination with complete pivoting -- the first improvement upon Wilkinson's original 1961 bound of $2 \, n ^{0.25\ln n +0.5}$.

Numerical Analysis
Symmetric matrix inversion using modified Gaussian elimination

Anton Kochnev,Nicolai Savelov

DOI: https://doi.org/10.48550/arXiv.1504.06734

2015-04-25

Mathematical Software

Abstract:In this paper we present two different variants of method for symmetric matrix inversion, based on modified Gaussian elimination. Both methods avoid computation of square roots and have a reduced machine time's spending. Further, both of them can be used efficiently not only for positive (semi-) definite, but for any non-singular symmetric matrix inversion. We use simulation to verify results, which represented in this paper.
Gaussian Elimination with Randomized Complete Pivoting

Christopher Melgaard,Ming Gu

DOI: https://doi.org/10.48550/arXiv.1511.08528

2015-11-26

Numerical Analysis

Abstract:Gaussian elimination with partial pivoting (GEPP) has long been among the most widely used methods for computing the LU factorization of a given matrix. However, this method is also known to fail for matrices that induce large element growth during the factorization process. In this paper, we propose a new scheme, Gaussian elimination with randomized complete pivoting (GERCP) for the efficient and reliable LU factorization of a given matrix. GERCP satisfies GECP (Gaussian elimination with complete pivoting) style element growth bounds with high probability, yet costs only marginally higher than GEPP. Our numerical experimental results strongly suggest that GERCP is as reliable as GECP and as efficient as GEPP for computing the LU factorization.
A New Proof on the Growth Factor in Gaussian Elimination for Generalized Higham Matrices

Qianping Guo,Houbiao Li

2014-01-01

Abstract:—The generalized Higham matrix is a complex symmetric matrix A = B + iC , where both B ∈ C n × n and C ∈ C n × n are Hermitian positive deﬁnite, and i = √− 1 is the imaginary unit. The growth factor in Gaussian elimination is less than 3 √ 2 for this kind of matrices. In this paper, we give a new brief proof on this result by different techniques, which can be understood very easily, and obtain some new ﬁndings.

A simple way to speedup Gauss Elimination

The Modified Solutoin for Large Sparse Symmetric Linear Systems in Electromagnetic Field Analysis

Approximate Gaussian Elimination for Laplacians: Fast, Sparse, and Simple

Average-case analysis of the Gaussian Elimination with Partial Pivoting

Cuda Based Parallel Computation For Gauss Elimination Method

Generalized elimination method and its applications to matrix factorizations

The asymptotic complexity of matrix reduction over finite fields

On the worst-case complexity of integer Gaussian elimination

Alternating iterative method based on modified Gauss-Seidel method

Gaussian elimination corrects pivoting mistakes

Speeding Up the GVW Algorithm Via a Substituting Method

Iterative methods for solving factorized linear systems

A modification of the Jacobi-Davidson method

Efficient Accelerated Coordinate Descent Methods and Faster Algorithms for Solving Linear Systems

An Efficient Gauss--Newton Algorithm for Symmetric Low-Rank Product Matrix Approximations

Good and Fast Row-Sparse ah-Symmetric Reflexive Generalized Inverses

A New Upper Bound For the Growth Factor in Gaussian Elimination with Complete Pivoting

Symmetric matrix inversion using modified Gaussian elimination

Gaussian Elimination with Randomized Complete Pivoting

A New Proof on the Growth Factor in Gaussian Elimination for Generalized Higham Matrices