A simpler DGMRES

Jieyong Zhou,Hui Wei

DOI: https://doi.org/10.1016/j.amc.2010.05.024

IF: 4.397

2010-01-01

Applied Mathematics and Computation

Abstract:DGMRES is an algorithm to solve the Drazin-inverse solution of large scale and sparse consistent or inconsistent singular linear systems with an arbitrary index. In this paper, we present a simpler version of this algorithm.

What problem does this paper attempt to address?

Stagnation Analysis Of Dgmres

Hy Zhou,Ym Wei

DOI: https://doi.org/10.1016/S0096-3003(03)00320-5

IF: 4.397

2004-01-01

Applied Mathematics and Computation

Abstract:We Study problems for which the iterative method DGMRES for solving Drazin inverse solution of singular linear systems of equations which makes no progress in its initial iterations. Particularly, our tool for analysis is nonlinear system of equations, the stagnation system, that characterize this behavior. For problems of dimension two and index one we can solve this system explicitly. We partially extend this result to higher dimensions for a class of eigenvector matrices called extreme. We also Study the real matrices. (C) 2003 Elsevier Inc. All rights reserved.
A two-step algorithm for solving singular linear systems with index one.

Jieyong Zhou,Yimin Wei

DOI: https://doi.org/10.1016/j.amc.2005.07.027

IF: 4.397

2006-01-01

Applied Mathematics and Computation

Abstract:In this paper, a two-step algorithm for solving singular linear systems is presented. We compare this algorithm with a DGCR type algorithm [A. Sidi, A unified approach to Krylov subspace methods for the Drazin-inverse solution of singular non-symmetric linear systems, Linear Algebra Appl. 298 (1999) 99–113] by numerical experiments. An error analysis is given.
A flexible and adaptive Simpler GMRES with deflated restarting for shifted linear systems

Hong-Xiu Zhong,Xian-Ming Gu

DOI: https://doi.org/10.1016/j.camwa.2019.03.017

2019-08-01

Abstract:In this paper, two efficient iterative algorithms based on the Simpler GMRES method are proposed for solving shifted linear systems. To make full use of the shifted structure, the proposed algorithms utilizing the deflated restarting strategy and flexible preconditioning can significantly reduce the number of matrix–vector products and the elapsed CPU time. Numerical experiments are reported to illustrate the performance and effectiveness of the proposed algorithms.

mathematics, applied
DFOM algorithm and error analysis for projection methods for solving singular linear system

Jieyong Zhou,Yimin Wei

DOI: https://doi.org/10.1016/j.amc.2003.08.036

IF: 4.397

2004-01-01

Applied Mathematics and Computation

Abstract:In this paper, DFOM algorithm is presented. We compare DGMRES algorithm with DFOM algorithm by numerical experiment. Error analysis for projection method is also given.
Obtaining the pseudoinverse solution of singular range-symmetric linear systems with GMRES-type methods

Kui Du,Jia-Jun Fan,Fang Wang

2024-01-23

Abstract:It is well known that for singular inconsistent range-symmetric linear systems, the generalized minimal residual (GMRES) method determines a least squares solution without breakdown. The reached least squares solution may be or not be the pseudoinverse solution. We show that a lift strategy can be used to obtain the pseudoinverse solution. In addition, we propose a new iterative method named RSMAR (minimum $\mathbf A$-residual) for range-symmetric linear systems $\mathbf A\mathbf x=\mathbf b$. At step $k$ RSMAR minimizes $\|\mathbf A\mathbf r_k\|$ in the $k$th Krylov subspace generated with $\{\mathbf A, \mathbf r_0\}$ rather than $\|\mathbf r_k\|$, where $\mathbf r_k$ is the $k$th residual vector and $\|\cdot\|$ denotes the Euclidean vector norm. We show that RSMAR and GMRES terminate with the same least squares solution when applied to range-symmetric linear systems. We provide two implementations for RSMAR. Our numerical experiments show that RSMAR is the most suitable method among GMRES-type methods for singular inconsistent range-symmetric linear systems.

Numerical Analysis
A Simpler GMRES and Its Adaptive Variant for Shifted Linear Systems.

Yan-Fei Jing,Pei Yuan,Ting-Zhu Huang

DOI: https://doi.org/10.1002/nla.2076

2016-01-01

Numerical Linear Algebra with Applications

Abstract:SummaryA variant of the simpler GMRES method is developed for solving shifted linear systems (SGMRES‐Sh), exhibiting almost the same advantage of the simpler GMRES method over the regular GMRES method. Because the remedy adapted by GMRES‐Sh is no longer feasible for SGMRES‐Sh due to the differences between simpler GMRES and GMRES for constructing the residual vectors of linear systems, we take an alternative strategy to force the residual vectors of the add system also be orthogonal to the subspaces, to which the residual vectors of the seed system are orthogonal when the seed system is solved with the simpler GMRES method. In addition, a seed selection strategy is also employed for solving the rest non‐converged linear systems. Furthermore, an adaptive version of SGMRES‐Sh is presented for the purpose of improving the stability of SGMRES‐Sh based on the technique of the adaptive choice of the Krylov subspace basis developed for the adaptive simpler GMRES. Numerical experiments demonstrate the benefits of the presented methods.
On IGMRES: an Incomplete Generalized Minimal Residual Method for Large Unsymmetric Linear Systems

Zhongxiao Jia

DOI: https://doi.org/10.1007/bf02882268

1998-01-01

Science in China Series A Mathematics

Abstract:The truncated version of the generalized minimal residual method (GMRES), the incomplete generalized minimal residual method (IGMRES), is studied. It is based on an incomplete orthogonalization of the Krylov vectors in question, and gives an approximate or quasi-minimum residual solution over the Krylov subspace. A convergence analysis of this method is given, showing that in the non-restarted version IGMRES can behave like GMRES once the basis vectors of Krylov subspace generated by the incomplete orthogonalization are strongly linearly independent. Meanwhile, some relationships between the residual norms for IOM and IGMRES are established. Numerical experiments are reported to show convergence behavior of IGMRES and of its restarted version IGMRES( m ).
A Block GMRES Method with Deflated Restarting for Solving Linear Systems with Multiple Shifts and Multiple Right-Hand Sides

Dong-Lin Sun,Ting-Zhu Huang,Yan-Fei Jing,Bruno Carpentieri

DOI: https://doi.org/10.1002/nla.2148

2018-01-01

Numerical Linear Algebra with Applications

Abstract:SummaryThe restarted block generalized minimum residual method (BGMRES) with deflated restarting (BGMRES‐DR) was proposed by Morgan to dump the negative effect of small eigenvalues from the convergence of the BGMRES method. More recently, Wu et al. introduced the shifted BGMRES method (BGMRES‐Sh) for solving the sequence of linear systems with multiple shifts and multiple right‐hand sides. In this paper, a new shifted block Krylov subspace algorithm that combines the characteristics of both the BGMRES‐DR and the BGMRES‐Sh methods is proposed. Moreover, our method is enhanced with a seed selection strategy to handle the case of almost linear dependence of the right‐hand sides. Numerical experiments illustrate the potential of the proposed method to solve efficiently the sequence of linear systems with multiple shifts and multiple right‐hand sides, with and without preconditioner, also against other state‐of‐the‐art solvers.
Block GMRES Method with Inexact Breakdowns and Deflated Restarting

E. Agullo,L. Giraud,Y.-F. Jing

DOI: https://doi.org/10.1137/140961912

IF: 1.908

2014-01-01

SIAM Journal on Matrix Analysis and Applications

Abstract:We consider the solution of large linear systems with multiple right-hand sides using a block GMRES approach. We introduce a new algorithm that effectively handles the situation of almost rank deficient block generated by the block Arnoldi procedure and that enables the recycling of spectral information at restart. The first feature is inherited from an algorithm introduced by Robbé and Sadkane [Linear Algebra Appl., 419 (2006), pp. 265--285], while the second one is obtained by extending the deflated restarting strategy proposed by Morgan [Appl. Numer. Math., 54 (2005), pp. 222--236]. Through numerical experiments, we show that the new algorithm combines efficiently the attractive numerical features of its two parents and outperforms them.

mathematics, applied
Flexible and deflated variants of the block shifted GMRES method

Dong-Lin Sun,Ting-Zhu Huang,Bruno Carpentieri,Yan-Fei Jing

DOI: https://doi.org/10.1016/j.cam.2018.05.053

IF: 2.872

2019-01-01

Journal of Computational and Applied Mathematics

Abstract:The solution of linear systems with multiple shifts and multiple right-hand sides given simultaneously is required in many large-scale scientific and engineering applications. In this paper we introduce new flexible and deflated variants of the shifted block GMRES method for this problem class. The proposed methods solve the whole sequence of linear systems simultaneously, detecting effectively the linear systems convergence and allowing the use of variable preconditioning which may be particularly useful in some applications. Numerical experiments are illustrated to show the overall significant robustness of the iterative method for solving general sparse multi-shifted and multiple right-hand-side systems, and in realistic PageRank calculations. To the best of our knowledge, this is the first Krylov subspace method that combines deflation techniques and variable preconditioning for solving sequences of multi-shifted linear systems with multiple right-hand sides simultaneously.

mathematics, applied
A Deflated Block Flexible Gmres-Dr Method for Linear Systems with Multiple Right-Hand Sides

Jing Meng,Pei-Yong Zhu,Hou-Biao Li,Xian-Ming Gu

2014-01-01

Abstract:This study is mainly focused on the iterative solution of multiple linear systems with several right-hand sides. To solve such systems efficiently, we first present a flexible version of block GMRES with deflation of eigenvalues according to [R. B. Morgan, Restarted block-GMRES with deflation of eigenvalues, Appl. Numer. Math., 54 (2005), pp. 222-236] and then apply a modified block Arnoldi vector deflation technique to accelerate the convergence of this new flexible version. Incorporating this deflation technique, the new algorithm can address the possible linear dependence at each iteration during the block Arnoldi procedure and reduce computational expense. Moreover, by analyzing its main mathematical properties, we show that the vector deflation procedure arises from the non-increasing behavior of the singular values of the block residual. In addition, the new approach also inherits the property of deflating small eigenvalues to mitigate convergence slowdown. Finally, the effectiveness of the proposed method is illustrated by some numerical experiments.
Deflated GMRES for systems with multiple shifts and multiple right-hand sides

Dean Darnell,Ronald B. Morgan,Walter Wilcox

DOI: https://doi.org/10.1016/j.laa.2008.04.019

IF: 1.307

2008-11-01

Linear Algebra and its Applications

Abstract:We consider solution of multiply shifted systems of nonsymmetric linear equations, possibly also with multiple right-hand sides. First, for a single right-hand side, the matrix is shifted by several multiples of the identity. Such problems arise in a number of applications, including lattice quantum chromodynamics where the matrices are complex and non-Hermitian. Some Krylov iterative methods such as GMRES and BiCGStab have been used to solve multiply shifted systems for about the cost of solving just one system. Restarted GMRES can be improved by deflating eigenvalues for matrices that have a few small eigenvalues. We show that a particular deflated method, GMRES-DR, can be applied to multiply shifted systems.In quantum chromodynamics, it is common to have multiple right-hand sides with multiple shifts for each right-hand side. We develop a method that efficiently solves the multiple right-hand sides by using a deflated version of GMRES and yet keeps costs for all of the multiply shifted systems close to those for one shift. An example is given showing this can be extremely effective with a quantum chromodynamics matrix.

mathematics, applied
A spectrally preconditioned and initially deflated variant of the restarted block GMRES method for solving multiple right-hand sides linear systems

Dong-Lin Sun,Bruno Carpentieri,Ting-Zhu Huang,Yan-Fei Jing

DOI: https://doi.org/10.1016/j.ijmecsci.2018.06.033

IF: 7.3

2018-08-01

International Journal of Mechanical Sciences

Abstract:The solution of large linear systems with multiple right-hand sides given simultaneously is required in many large-scale scientific and engineering applications modelled by either partial differential or boundary integral equations. Block Krylov subspace methods are attractive to use for this problem class as they can overcome the memory bottleneck of direct methods and they perform block matrix-vector products, achieving high computational efficiency on modern cache-based computer architectures. In this paper we introduce variants of the block GMRES method that combine initial deflation and eigenvalue recycling strategies to remedy some typical convergence problems of block Krylov solvers. The new class of block iterative solvers has the ability to handle the approximate linear dependence of the block of right-hand sides and exploits approximate invariant subspaces recycled over the iterations to mitigate the bad effects that small eigenvalues can have on the convergence, by adapting an existing preconditioner. We illustrate the numerical behavior of the spectrally updated and initially deflated block GMRES method on a set of linear systems arising from the discretization of the Dirac equation and of boundary integral equations in electromagnetics scattering.

engineering, mechanical,mechanics
Preprocessed GMRES for fast solution of linear equations

Juan Zhang,Yiyi Luo

2024-04-09

Abstract:The article mainly introduces preprocessing algorithms for solving linear equation systems. This algorithm uses three algorithms as inner iterations, namely RPCG algorithm, ADI algorithm, and Kaczmarz algorithm. Then, it uses BA-GMRES as an outer iteration to solve the linear equation system. These three algorithms can indirectly generate preprocessing matrices, which are used for solving equation systems. In addition, we provide corresponding convergence analysis and numerical examples. Through numerical examples, we demonstrate the effectiveness and feasibility of these preprocessing methods. Furthermore, in the Kaczmarz algorithm, we introduce both constant step size and adaptive step size, and extend the parameter range of the Kaczmarz algorithm to $\alpha\in(0,\infty)$. We also study the solution rate of linear equation systems using different step sizes. Numerical examples show that both constant step size and adaptive step size have higher solution efficiency than the solving algorithm without preprocessing.

Numerical Analysis
Gradient Methods for Computing the Drazin-inverse Solution

Sladjana Miljkovic,Marko Miladinovic,Predrag S. Stanimirovic,Yimin Wei

DOI: https://doi.org/10.1016/j.cam.2013.04.030

IF: 2.872

2013-01-01

Journal of Computational and Applied Mathematics

Abstract:The problem of computing the vector A^Db, in the cases when it provides a solution of a singular linear system, is reduced to the problem of finding a least-squares solution of an appropriate linear system. Later the presented ideas are extended to arbitrary linear systems. As a consequence, we present iterative methods for computing the vector A^Db in general. The functionality of the exposed algorithms is based on a specific representation of the Drazin-inverse solution, as well as the properties that we introduce. Convergence results of the proposed methods are also considered. Illustrative numerical examples are presented.
A Note On The Representations For The Drazin Inverse Of 2x2 Block Matrices

Xiezhang Li,Yimin Wei

DOI: https://doi.org/10.1016/j.laa.2007.01.005

IF: 1.307

2007-01-01

Linear Algebra and its Applications

Abstract:In 1979, Campbell and Meyer proposed the problem of finding a formula for the Drazin inverse of a 2 x 2 matrix M =[GRAPHICS]in terms of its various blocks, where the blocks A and D are required to be square matrices. Special cases of the problems have been studied. In particular, a representation of the Drazin inverse of M, denoted by M-D, has recently been obtained under the assumptions that C(I - AA(D))B = O and A(I - AA(D))B = O together with the condition that the generalized Schur complement D - CA(D)B be either nonsingular or zero. We derive an alternative representation for M-D under the same assumptions, but with the condition on the Schur complement in the hypothesis replaced by the condition that R(CAA(D)) subset of N(B) boolean AND N(D), where R(.) and N(.) are the range and null space of a matrix. (C) 2007 Elsevier Inc. All rights reserved.
Three double multi-step randomized extended Kaczmarz methods for solving large sparse inconsistent linear systems

Zhi-Min Xu,Hai-Long Shen,Xin-Hui Shao

DOI: https://doi.org/10.1007/s40314-024-02819-y

2024-10-25

Computational and Applied Mathematics

Abstract:In order to solve large sparse inconsistent linear equations more efficiently and accurately, based on the multi-step randomized extended Kaczmarz method, we propose three double multi-step randomized extended Kaczmarz algorithms. Firstly, we introduce the double multi-step randomized extended Kaczmarz (DMREK) algorithm with probability criteria, where the selection of parameters is based on the order of coefficient matrix, and the convergence of this algorithm has been proven. Additionally, we propose two new algorithms, namely the double randomized multi-step randomized extended Kaczmarz (DRMREK) algorithm with random parameters and the residual-based randomized multi-step randomized extended Kaczmarz (RRMREK) algorithm. Numerical experimental results demonstrate that the three new algorithms DMREK, DRMREK, and RRMREK perform well. And the RRMREK algorithm, along with others, demonstrates advantages in terms of the number of iteration steps.

mathematics, applied
Low-rank generalized alternating direction implicit iteration method for solving matrix equations

Juan Zhang,Wenlu Xun

DOI: https://doi.org/10.1007/s40314-024-02774-8

2024-05-29

Computational and Applied Mathematics

Abstract:This paper presents an effective low-rank generalized alternating direction implicit iteration (R-GADI) method for solving large-scale sparse and stable Lyapunov matrix equations and continuous-time algebraic Riccati matrix equations. The method is based on generalized alternating direction implicit iteration (GADI), which exploits the low-rank property of matrices and utilizes the Cholesky factorization approach for solving. The advantage of the new algorithm lies in its direct and efficient low-rank formulation, which is a variant of the Cholesky decomposition in the Lyapunov GADI method, saving storage space and making it computationally effective. When solving the continuous-time algebraic Riccati matrix equation, the Riccati equation is first simplified to a Lyapunov equation using the Newton method, and then the R-GADI method is employed for computation. Additionally, we analyze the convergence of the R-GADI method and prove its consistency with the convergence of the GADI method. Finally, the effectiveness of the new algorithm is demonstrated through corresponding numerical experiments.

mathematics, applied
Preconditioning Low Rank Generalized Minimal Residual Method (GMRES) for Implicit Discretizations of Matrix Differential Equations

Shixu Meng,Daniel Appelo,Yingda Cheng

2024-10-10

Abstract:This work proposes a new class of preconditioners for the low rank Generalized Minimal Residual Method (GMRES) for multiterm matrix equations arising from implicit timestepping of linear matrix differential equations. We are interested in computing low rank solutions to matrix equations, e.g. arising from spatial discretization of stiff partial differential equations (PDEs). The low rank GMRES method is a particular class of Krylov subspace method where the iteration is performed on the low rank factors of the solution. Such methods can exploit the low rank property of the solution to save on computational and storage cost. Of critical importance for the efficiency and applicability of the low rank GMRES method is the availability of an effective low rank preconditioner that operates directly on the low rank factors of the solution and that can limit the iteration count and the maximal Krylov rank. The preconditioner we propose here is based on the basis update and Galerkin (BUG) method, resulting from the dynamic low rank approximation. It is a nonlinear preconditioner for the low rank GMRES scheme that naturally operates on the low rank factors. Extensive numerical tests show that this new preconditioner is highly efficient in limiting iteration count and maximal Krylov rank. We show that the preconditioner performs well for general diffusion equations including highly challenging problems, e.g. high contrast, anisotropic equations. Further, it compares favorably with the state of the art exponential sum preconditioner. We also propose a hybrid BUG - exponential sum preconditioner based on alternating between the two preconditioners.

Numerical Analysis

A simpler DGMRES

Stagnation Analysis Of Dgmres

A two-step algorithm for solving singular linear systems with index one.

A flexible and adaptive Simpler GMRES with deflated restarting for shifted linear systems

DFOM algorithm and error analysis for projection methods for solving singular linear system

Obtaining the pseudoinverse solution of singular range-symmetric linear systems with GMRES-type methods

A Simpler GMRES and Its Adaptive Variant for Shifted Linear Systems.

On IGMRES: an Incomplete Generalized Minimal Residual Method for Large Unsymmetric Linear Systems

A Block GMRES Method with Deflated Restarting for Solving Linear Systems with Multiple Shifts and Multiple Right-Hand Sides

Block GMRES Method with Inexact Breakdowns and Deflated Restarting

Flexible and deflated variants of the block shifted GMRES method

A Deflated Block Flexible Gmres-Dr Method for Linear Systems with Multiple Right-Hand Sides

Deflated GMRES for systems with multiple shifts and multiple right-hand sides

A spectrally preconditioned and initially deflated variant of the restarted block GMRES method for solving multiple right-hand sides linear systems

Preprocessed GMRES for fast solution of linear equations

Gradient Methods for Computing the Drazin-inverse Solution

A Note On The Representations For The Drazin Inverse Of 2x2 Block Matrices

Three double multi-step randomized extended Kaczmarz methods for solving large sparse inconsistent linear systems

Low-rank generalized alternating direction implicit iteration method for solving matrix equations

Preconditioning Low Rank Generalized Minimal Residual Method (GMRES) for Implicit Discretizations of Matrix Differential Equations