Restarted weighted full orthogonalization method for shifted linear systems

JingYan-Fei,HuangTing-Zhu

IF: 3.218

2009-01-01

Computers & Mathematics with Applications

Abstract:It is known that the restarted full orthogonalization method (FOM) outperforms the restarted generalized minimum residual method (GMRES) in several circumstances for solving shifted linear systems ...

What problem does this paper attempt to address?

Restarted Weighted Full Orthogonalization Method for Shifted Linear Systems

Yan-Fei Jing,Ting-Zhu Huang

DOI: https://doi.org/10.1016/j.camwa.2008.10.088

IF: 3.218

2009-01-01

Computers & Mathematics with Applications

Abstract:It is known that the restarted full orthogonalization method (FOM) outperforms the restarted generalized minimum residual method (GMRES) in several circumstances for solving shifted linear systems when the shifts are handled simultaneously. On the basis of the Weighted Arnoldi process, a weighted version of the Restarted Shifted FOM is proposed, which can provide accelerating convergence rate with respect to the number of restarts. In the cases where our hybrid algorithm needs less enough number of restarts to converge than the Restarted Shifted FOM, the associated CPU consuming time is also reduced, as shown by the numerical experiments. Moreover, our algorithm is able to solve certain shifted systems which the Restarted Shifted FOM cannot handle sometimes.
Restarted Hessenberg Method for Solving Shifted Nonsymmetric Linear Systems

Xian-Ming Gu,Ting-Zhu Huang,Guojian Yin,Bruno Carpentieri,Chun Wen,Lei Du

DOI: https://doi.org/10.1016/j.cam.2017.09.047

IF: 2.872

2017-01-01

Journal of Computational and Applied Mathematics

Abstract:It is known that the restarted full orthogonalization method (FOM) outperforms the restarted generalized minimum residual (GMRES) method in several circumstances for solving shifted linear systems when the shifts are handled simultaneously. Many variants of them have been proposed to enhance their performance. We show that another restarted method, the restarted Hessenberg method (Heyouni, 1996) based on Hessenberg procedure, can effectively be employed, which can provide accelerating convergence rate with respect to the number of restarts. Theoretical analysis shows that the new residual of shifted restarted Hessenberg method is still collinear with each other. In these cases where the proposed algorithm needs less enough elapsed CPU time to converge than the earlier established restarted shifted FOM, the weighted restarted shifted FOM, and some other popular shifted iterative solvers based on the short-term vector recurrence, as shown via extensive numerical experiments involving the recently popular application of handling time fractional differential equations. (C) 2017 Elsevier B.V. All rights reserved.
Restarted Hessenberg Method for Shifted Nonsymmetric Linear Systems with Applications to Fractional Differential Equations

Xian‐Ming Gu,Ting‐Zhu Huang,Guojian Yin,Bruno Carpentieri,Chun Wen

2015-01-01

Abstract:It is known that the restarted full orthogonalization method (FOM) outperforms the restarted generalized minimum residual method (GMRES) in several circumstances for solving shifted linear systems when the shifts are handled simultaneously. Variants of them have been proposed to enhance their performance. We show that another restarted method, restarted Hessenberg method [M. Heyouni, Methode de Hessenberg Generalisee et Applications, Ph.D. Thesis, Universite des Sciences et Technologies de Lille, France, 1996] based on Hessenberg process, can effectively be employed, which can provide accelerating convergence rate with respect to the number of restarts. Theoretical analysis shows that the new residual of shifted restarted Hessenberg reduction method is still collinear with each other. In these cases where our proposed algorithm needs less enough number of restarts to converge than the earlier established restarted shifted FOM and weighted restarted shifted FOM, the associated CPU consuming time is also considerably reduced, as shown via extensive numerical experiments involving the recent popular applications of handling structural dynamics, time-fractional convection-diffusion equations and space-fractional diffusion equations.
Near-optimal convergence of the full orthogonalization method

Tyler Chen,Gérard Meurant

2024-08-27

Abstract:We establish a near-optimality guarantee for the full orthogonalization method (FOM), showing that the overall convergence of FOM is nearly as good as GMRES. In particular, we prove that at every iteration $k$, there exists an iteration $j\leq k$ for which the FOM residual norm at iteration $j$ is no more than $\sqrt{k+1}$ times larger than the GMRES residual norm at iteration $k$. This bound is sharp, and it has implications for algorithms for approximating the action of a matrix function on a vector.\end{abstract}

Numerical Analysis
On restarted and deflated block FOM and GMRES methods for sequences of shifted linear systems

Lakhdar Elbouyahyaoui,Mohammed Heyouni,Azita Tajaddini,Farid Saberi-Movahed

DOI: https://doi.org/10.1007/s11075-020-01007-3

IF: 2.37

2020-09-09

Numerical Algorithms

Abstract:The problem of shifted linear systems is an important and challenging issue in a number of research applications. Krylov subspace methods are effective techniques for different kinds of this problem due to their advantages in large and sparse matrix problems. In this paper, two new block projection methods based on respectively block FOM and block GMRES are introduced for solving sequences of shifted linear systems. We first express the original problem explicitly by a sequence of Sylvester matrix equations whose coefficient matrices are obtained from the shifted linear systems. Then, we show the restarted shifted block FOM (rsh-BFOM) method and derive some of its properties. We also present a framework for the restarted shifted block GMRES (rsh-BGMRES) method. In this regard, we describe two variants of rsh-BGMRES, including (1) rsh-BGMRES with an unshifted base system that applies a fixed unshifted base system and (2) rsh-BGMRES with a variable shifted base system in which the base block system can change after restart. Furthermore, we consider the use of deflation techniques for improving the performance of the rsh-BFOM and rsh-BGMRES methods. Finally, some numerical experiments are conducted to demonstrate the effectiveness of the proposed methods.

mathematics, applied
Restarted Global FOM and GMRES Algorithms for the Stein-like Matrix Equationx+m(x)=c

Sheng-Kun Li,Ting-Zhu Huang

DOI: https://doi.org/10.1016/j.amc.2018.11.058

IF: 4.397

2019-01-01

Applied Mathematics and Computation

Abstract:In this paper, we propose the restarted global full orthogonalization method (Gl-FOM) and global generalized minimum residual (Gl-GMRES) method to solve the Stein-like matrix equation X + M ( X ) = C with M ( X ) = A X B , M ( X ) = A X ⊤ B , M ( X ) = A X ¯ B or M ( X ) = A X H B , respectively, where X is an unknown matrix to be solved. First, by using a real inner product in complex matrix spaces, a generalized global Arnoldi process is given. Then we demonstrate how to employ the restarted Gl-FOM and Gl-GMRES algorithms for solving the Stein-like matrix equation. The proposed algorithms take advantage of the shifted structure of the matrix equation and are implemented by the original coefficient matrices. Finally, some numerical examples are given to illustrate the effectiveness with comparison to some existing methods.
Global GMRES with Deflated Restarting for Families of Shifted Linear Systems

Jing Meng,Peiyong Zhu,Hou-Biao Li

DOI: https://doi.org/10.5281/zenodo.1097489

2014-01-01

Abstract:Many problems in science and engineering field require the solution of shifted linear systems with multiple right hand sides and multiple shifts. To solve such systems efficiently, the implicitly restarted global GMRES algorithm is extended in this paper. However, the shift invariant property could no longer hold over the augmented global Krylov subspace due to adding the harmonic Ritz matrices. To remedy this situation, we enforce the collinearity condition on the shifted system and propose shift implicitly restarted global GMRES. The new method not only improves the convergence but also has a potential to simultaneously compute approximate solution for the shifted systems using only as many matrix vector multiplications as the solution of the seed system requires. In addition, some numerical experiments also confirm the effectiveness of our method. Keywords—Shifted linear systems, global Krylov subspace, GLGMRESIR, GLGMRESIRsh, harmonic Ritz matrix, harmonic
On IOM(q): the Incomplete Orthogonalization Method for Large Unsymmetric Linear Systems

ZX Jia

DOI: https://doi.org/10.1002/(sici)1099-1506(199611/12)3:6<491::aid-nla87>3.0.co;2-9

1996-01-01

Numerical Linear Algebra with Applications

Abstract:The incomplete orthogonalization method (IOM(q)), a truncated version of the full orthogonalization method (FOM) proposed by Saad, has been used for solving large unsymmetric linear systems. However, no convergence analysis has been given. In this paper, IOM(q) is analysed in detail from a theoretical point of view. A number of important results are derived showing how the departure of the matrix A from symmetry affects the basis vectors generated by IOM(q), and some relationships between the residuals for IOM(q) and FOM are established. The results show that IOM(q) behaves much like FOM once the basis vectors generated by it are well conditioned. However, it is proved that IOM(q) may generate an ill-conditioned basis for a general unsymmetric matrix such that IOM(q) may fail to converge or at least cannot behave like FOM. Owing to the mathematical equivalence between IOM(q) and the truncated ORTHORES(q) developed by Young and Jea, insights are given into the convergence of the latter. A possible strategy is proposed for choosing the parameter q involved in IOM(q). Numerical experiments are reported to show convergence behaviour of IOM(q) and of its restarted version.
Solving Shifted Linear Systems with Restarted GMRES Augmented with Error Approximations.

Rui-Rui Wang,Qiang Niu,Xiao-Bin Tang,Xiang Wang

DOI: https://doi.org/10.1016/j.camwa.2019.03.037

IF: 3.218

2019-01-01

Computers & Mathematics with Applications

Abstract:In this paper, we investigate a variant of the restarted GMRES method for solving a series of large sparse linear systems. Restarting is carried out by augmenting Krylov subspaces with some recently generated error approximations from the seed system. The method can preserve a nice property that allows solving the seed and the added linear systems at the cost of only one matrix–vector multiplication per iteration. Compared with solving each added linear system separately, the advantage of the new scheme is to lower down the overall cost of solving all added linear systems. Numerical experiments illustrate the efficiency of the acceleration strategy.
A flexible and adaptive Simpler GMRES with deflated restarting for shifted linear systems

Hong-Xiu Zhong,Xian-Ming Gu

DOI: https://doi.org/10.1016/j.camwa.2019.03.017

2019-08-01

Abstract:In this paper, two efficient iterative algorithms based on the Simpler GMRES method are proposed for solving shifted linear systems. To make full use of the shifted structure, the proposed algorithms utilizing the deflated restarting strategy and flexible preconditioning can significantly reduce the number of matrix–vector products and the elapsed CPU time. Numerical experiments are reported to illustrate the performance and effectiveness of the proposed algorithms.

mathematics, applied
Structure Preserving Balanced Proper Orthogonal Decomposition for Second-Order Form Systems Via Shifted Legendre Polynomials

Zhi-Hua Xiao,Yao-Lin Jiang,Zhen-Zhong Qi

DOI: https://doi.org/10.1049/iet-cta.2018.5946

IF: 2.67

2019-01-01

IET Control Theory and Applications

Abstract:This study considers structure preserving balanced proper orthogonal decomposition for second-order form systems via shifted Legendre polynomials. The proposed approach is to use time interval empirical Gramians of the first-order representation, which are constructed from impulse responses by solving block tridiagonal linear systems, to generate approximate balanced system for the large-scale second-order form system. The balancing transformation is directly computed from the expansion coefficients of impulse responses in the space spanned by shifted Legendre polynomials, without computing the full Gramians for the first-order representation. Then, the reduced second-order model is constructed by truncating the states corresponding to the small approximate singular values. Furthermore, in combination with the dominant subspace projection method, the authors modify the reduction procedure to alleviate the shortcoming of the above method, which may unexpectedly lead to unstable systems even though the original one is stable. The stability preservation of the reduced model is briefly discussed. Finally, the effectiveness of the proposed methods is demonstrated by two numerical experiments.
On IGMRES: an Incomplete Generalized Minimal Residual Method for Large Unsymmetric Linear Systems

Zhongxiao Jia

DOI: https://doi.org/10.1007/bf02882268

1998-01-01

Science in China Series A Mathematics

Abstract:The truncated version of the generalized minimal residual method (GMRES), the incomplete generalized minimal residual method (IGMRES), is studied. It is based on an incomplete orthogonalization of the Krylov vectors in question, and gives an approximate or quasi-minimum residual solution over the Krylov subspace. A convergence analysis of this method is given, showing that in the non-restarted version IGMRES can behave like GMRES once the basis vectors of Krylov subspace generated by the incomplete orthogonalization are strongly linearly independent. Meanwhile, some relationships between the residual norms for IOM and IGMRES are established. Numerical experiments are reported to show convergence behavior of IGMRES and of its restarted version IGMRES( m ).
A Refined Second-order Arnoldi (RSOAR) Method for the Quadratic Eigenvalue Problem and Implicit Restarting

Zhongxiao Jia,Yuquan Sun

2010-01-01

Abstract:Based on the orthogonal projection principle and a second-order Arnoldi procedure that generates an orthonormal basis of the second-order Krylov subspace, the second-order Arnoldi (SOAR) method has been proposed for the quadratic eigenvalue problem. It can be used to compute some eigenpairs of the quadratic eigenvalue problem and preserve physical structures of the original problem. This paper proposes a refined second-order Arnoldi (RSOAR) method that is based on the refined projection principle and has the same structure preserving properties. To make the methods practical, restarting is generally necessary. However, the SOAR based methods cannot be restarted effectively. As the first step, to correct this serious deficiency, we instead use a modified SOAR procedure starting with a general vector. It is well known that if deflation occurs then implicit restarting cannot be applied to the MSOAR procedure. To this end, we propose an effective strategy to treat the critical deflation issue in the MSOAR procedure so that implicit restarting is applicable. We then develop implicitly restarted MSOAR and RSOAR algorithms, in which we propose the exact and refined shifts for the two algorithms, respectively. Numerical examples illustrate the efficiency of the restarted algorithms and the superiority of the restarted RSOAR algorithm.
Restarted GMRES Method Augmented with the Combination of Harmonic Ritz Vectors and Error Approximations

Qiang Niu,Linzhang Lu

DOI: https://doi.org/10.5281/zenodo.1061114

2012-01-01

Abstract:Restarted GMRES methods augmented with approximate eigenvectors are widely used for solving large sparse linear systems. Recently a new scheme of augmenting with error approximations is proposed. The main aim of this paper is to develop a restarted GMRES method augmented with the combination of harmonic Ritz vectors and error approximations. We demonstrate that the resulted combination method can gain the advantages of two approaches: (i) effectively deflate the small eigenvalues in magnitude that may hamper the convergence of the method and (ii) partially recover the global optimality lost due to restarting. The effectiveness and efficiency of the new method are demonstrated through various numerical examples. Keywords—Arnoldi process, GMRES, Krylov subspace, Systems of Linear equations
Global Convergence of Triangularized Orthogonalization-free Method

Weiguo Gao,Yingzhou Li,Bichen Lu

DOI: https://doi.org/10.4310/cms.2023.v21.n1.a9

2022-01-01

Communications in Mathematical Sciences

Abstract:This paper proves the global convergence of a triangularized orthogonalization-free method (TriOFM). TriOFM, in general, applies a triangularization idea to the gradient of an objective function and removes the rotation invariance in minimizers. More precisely, in this paper, the TriOFM works as an eigensolver for sizeable sparse matrices and obtains eigenvectors without any orthogonalization step. Due to the triangularization, the iteration is a discrete-time flow in a non-conservative vector field. The global convergence relies on the stable manifold theorem, whereas the convergence to stationary points is proved in detail in this paper. We provide two proofs inspired by the noisy power method and the noisy optimization method, respectively.
Global FOM and GMRES algorithms for a class of complex matrix equations.

Sheng-Kun Li,Ting-Zhu Huang

DOI: https://doi.org/10.1016/j.cam.2017.11.041

IF: 2.872

2018-01-01

Journal of Computational and Applied Mathematics

Abstract:In this paper, a generalized global Arnoldi process is given to produce an F-orthonormal basis of complex matrix space. Then we propose the global full orthogonalization method (Gl-FOM) and global generalized minimum residual (Gl-GMRES) method for solving a class of complex matrix equations. The new methods can be implemented by the original coefficient matrices. By using Schur complement, we give the expressions of the approximate solutions and the corresponding residuals. Some convergence results of the proposed methods are also derived. Finally, numerical examples are given to illustrate the effectiveness with comparison to other existing methods.
A Block GMRES Method with Deflated Restarting for Solving Linear Systems with Multiple Shifts and Multiple Right-Hand Sides

Dong-Lin Sun,Ting-Zhu Huang,Yan-Fei Jing,Bruno Carpentieri

DOI: https://doi.org/10.1002/nla.2148

2018-01-01

Numerical Linear Algebra with Applications

Abstract:SummaryThe restarted block generalized minimum residual method (BGMRES) with deflated restarting (BGMRES‐DR) was proposed by Morgan to dump the negative effect of small eigenvalues from the convergence of the BGMRES method. More recently, Wu et al. introduced the shifted BGMRES method (BGMRES‐Sh) for solving the sequence of linear systems with multiple shifts and multiple right‐hand sides. In this paper, a new shifted block Krylov subspace algorithm that combines the characteristics of both the BGMRES‐DR and the BGMRES‐Sh methods is proposed. Moreover, our method is enhanced with a seed selection strategy to handle the case of almost linear dependence of the right‐hand sides. Numerical experiments illustrate the potential of the proposed method to solve efficiently the sequence of linear systems with multiple shifts and multiple right‐hand sides, with and without preconditioner, also against other state‐of‐the‐art solvers.
A Parallel Reduced-Space Interior Point Method With Orthogonal Collocation for First-Swing Stability Constrained Emergency Control

Quanyuan Jiang,Yun Wang,Guangchao Geng

DOI: https://doi.org/10.1109/TPWRS.2013.2275175

2014-01-01

Abstract:First-swing stability constrained emergency control (FSCEC) enhances power system transient stability during large disturbances, but is difficult to solve for larger systems because of its computational complexity. The method proposed in this work guarantees first swing transient stability by using a parallel reduced-space interior point method (IPM) with orthogonal collocation to solve FSCEC problems. This novel algorithm discretizes differential-algebraic equations using orthogonal collocation, which leads to a relatively low problem dimension, and accelerates the optimization process through a reduced-space technique by utilizing the property of small degrees of freedom after numerical discretization. Furthermore, a two-level parallelism is explored in reduced-space IPM (RIPM) algorithm and implemented with state-of-the-art parallelization techniques. The proposed approach was benchmarked on a Beowulf cluster with 64 CPU cores to show its excellent computational efficiency.
Gl-QFOM and Gl-QGMRES: two efficient algorithms for quaternion linear systems with multiple right-hand sides

Tao Li,Qing-Wen Wang,Xin-Fang Zhang

2023-08-25

Abstract:In this paper, we propose the global quaternion full orthogonalization (Gl-QFOM) and global quaternion generalized minimum residual (Gl-QGMRES) methods, which are built upon global orthogonal and oblique projections onto a quaternion matrix Krylov subspace, for solving quaternion linear systems with multiple right-hand sides. We first develop the global quaternion Arnoldi procedure to preserve the quaternion Hessenberg form during the iterations. We then establish the convergence analysis of the proposed methods, and show how to apply them to solve the Sylvester quaternion matrix equation. Numerical examples are provided to illustrate the effectiveness of our methods compared with the traditional Gl-FOM and Gl-GMRES iterations for the real representations of the original linear systems.

Numerical Analysis
Optimal Solutions of Well-Posed Linear Systems via Low-Precision Right-Preconditioned GMRES with Forward and Backward Stabilization

Xiangmin Jiao

2023-03-08

Abstract:Linear systems in applications are typically well-posed, and yet the coefficient matrices may be nearly singular in that the condition number $\kappa(\boldsymbol{A})$ may be close to $1/\varepsilon_{w}$, where $\varepsilon_{w}$ denotes the unit roundoff of the working precision. It is well known that iterative refinement (IR) can make the forward error independent of $\kappa(\boldsymbol{A})$ if $\kappa(\boldsymbol{A})$ is sufficiently smaller than $1/\varepsilon_{w}$ and the residual is computed in higher precision. We propose a new iterative method, called Forward-and-Backward Stabilized Minimal Residual or FBSMR, by conceptually hybridizing right-preconditioned GMRES (RP-GMRES) with quasi-minimization. We develop FBSMR based on a new theoretical framework of essential-forward-and-backward stability (EFBS), which extends the backward error analysis to consider the intrinsic condition number of a well-posed problem. We stabilize the forward and backward errors in RP-GMRES to achieve EFBS by evaluating a small portion of the algorithm in higher precision while evaluating the preconditioner in lower precision. FBSMR can achieve optimal accuracy in terms of both forward and backward errors for well-posed problems with unpolluted matrices, independently of $\kappa(\boldsymbol{A})$. With low-precision preconditioning, FBSMR can reduce the computational, memory, and energy requirements over direct methods with or without IR. FBSMR can also leverage parallelization-friendly classical Gram-Schmidt in Arnoldi iterations without compromising EFBS. We demonstrate the effectiveness of FBSMR using both random and realistic linear systems.

Numerical Analysis

Restarted weighted full orthogonalization method for shifted linear systems

Restarted Weighted Full Orthogonalization Method for Shifted Linear Systems

Restarted Hessenberg Method for Solving Shifted Nonsymmetric Linear Systems

Restarted Hessenberg Method for Shifted Nonsymmetric Linear Systems with Applications to Fractional Differential Equations

Near-optimal convergence of the full orthogonalization method

On restarted and deflated block FOM and GMRES methods for sequences of shifted linear systems

Restarted Global FOM and GMRES Algorithms for the Stein-like Matrix Equationx+m(x)=c

Global GMRES with Deflated Restarting for Families of Shifted Linear Systems

On IOM(q): the Incomplete Orthogonalization Method for Large Unsymmetric Linear Systems

Solving Shifted Linear Systems with Restarted GMRES Augmented with Error Approximations.

A flexible and adaptive Simpler GMRES with deflated restarting for shifted linear systems

Structure Preserving Balanced Proper Orthogonal Decomposition for Second-Order Form Systems Via Shifted Legendre Polynomials

On IGMRES: an Incomplete Generalized Minimal Residual Method for Large Unsymmetric Linear Systems

A Refined Second-order Arnoldi (RSOAR) Method for the Quadratic Eigenvalue Problem and Implicit Restarting

Restarted GMRES Method Augmented with the Combination of Harmonic Ritz Vectors and Error Approximations

Global Convergence of Triangularized Orthogonalization-free Method

Global FOM and GMRES algorithms for a class of complex matrix equations.

A Block GMRES Method with Deflated Restarting for Solving Linear Systems with Multiple Shifts and Multiple Right-Hand Sides

A Parallel Reduced-Space Interior Point Method With Orthogonal Collocation for First-Swing Stability Constrained Emergency Control

Gl-QFOM and Gl-QGMRES: two efficient algorithms for quaternion linear systems with multiple right-hand sides

Optimal Solutions of Well-Posed Linear Systems via Low-Precision Right-Preconditioned GMRES with Forward and Backward Stabilization