On Analysis Of Projection Methods For Rational Function Approximation To The Matrix Exponential

Gang Wu,Yimin Wei

DOI: https://doi.org/10.1137/090758660

IF: 3.039

2010-01-01

SIAM Journal on Numerical Analysis

Abstract:We revisit the analysis of projection methods for rational function approximation to the matrix exponential, and present a new result for establishing convergence of the Pade-plus-Krylov approximation when parallel to A parallel to > 1.

What problem does this paper attempt to address?

Convergence Analysis of Projected Gradient Descent for Schatten-p Nonconvex Matrix Recovery

Yun Cai,Song Li

DOI: https://doi.org/10.1007/s11425-014-4949-1

2014-01-01

Science China Mathematics

Abstract:The matrix rank minimization problem arises in many engineering applications. As this problem is NP-hard, a nonconvex relaxation of matrix rank minimization, called the Schatten-p quasi-norm minimization (0 < p < 1), has been developed to approximate the rank function closely. We study the performance of projected gradient descent algorithm for solving the Schatten-p quasi-norm minimization (0 < p < 1) problem. Based on the matrix restricted isometry property (M-RIP), we give the convergence guarantee and error bound for this algorithm and show that the algorithm is robust to noise with an exponential convergence rate.
A Theoretical Comparison of Two Classes of Projection Methods

Zhongxiao Jia

2001-01-01

Abstract:We are concerned with conventional and refined projection methods for computing approximations to some eigenpairs (λ, ) of a large matrix . Given a subspace that contains an approximation to , these two methods compute approximations and to (λ, ), respectively. We establish three results. First, the refined Ritz vector is unique as the deviation of from approaches zero if λ is simple. Second, in terms of residual norm of the refined approximate eigenpair, we derive lower and upper bounds for the sine of the angle between the Ritz vector and the refined Ritz vector , and we prove that unless . Third, we establish relationships between the residual norm of the conventional methods and the residual norm of the refined methods, and we show that the latter is always smaller than the former if is not an exact eigenpair of .
Convergence Rate Analysis of a Dykstra-Type Projection Algorithm

Xiaozhou Wang,Ting Kei Pong

DOI: https://doi.org/10.1137/23m1545781

IF: 2.763

2024-02-07

SIAM Journal on Optimization

Abstract:SIAM Journal on Optimization, Volume 34, Issue 1, Page 563-589, March 2024. Given closed convex sets [math], [math], and some nonzero linear maps [math], [math], of suitable dimensions, the multiset split feasibility problem aims at finding a point in [math] based on computing projections onto [math] and multiplications by [math] and [math]. In this paper, we consider the associated best approximation problem, i.e., the problem of computing projections onto [math]; we refer to this problem as the best approximation problem in multiset split feasibility settings (BA-MSF). We adapt the Dykstra's projection algorithm, which is classical for solving the BA-MSF in the special case when all [math], to solve the general BA-MSF. Our Dykstra-type projection algorithm is derived by applying (proximal) coordinate gradient descent to the Lagrange dual problem, and it only requires computing projections onto [math] and multiplications by [math] and [math] in each iteration. Under a standard relative interior condition and a genericity assumption on the point we need to project, we show that the dual objective satisfies the Kurdyka-Łojasiewicz property with an explicitly computable exponent on a neighborhood of the (typically unbounded) dual solution set when each [math] is [math]-cone reducible for some [math]: this class of sets covers the class of [math]-cone reducible sets, which include all polyhedrons, second-order cone, and the cone of positive semidefinite matrices as special cases. Using this, explicit convergence rate (linear or sublinear) of the sequence generated by the Dykstra-type projection algorithm is derived. Concrete examples are constructed to illustrate the necessity of some of our assumptions.

mathematics, applied
Convergence analysis of inexact proximal point algorithms on Hadamard manifolds

Jinhua Wang,Chong Li,Genaro Lopez,Jen-Chih Yao

DOI: https://doi.org/10.1007/s10898-014-0182-2

2014-01-01

Journal of Global Optimization

Abstract:Inexact proximal point methods are extended to find singular points for multivalued vector fields on Hadamard manifolds. Convergence criteria are established under some mild conditions. In particular, in the case of proximal point algorithm, that is, ε _n=0 for each n , our results improve sharply the corresponding results in Li et al. ( 2009 ). Applications to optimization problems, variational inequality problems and gradient methods are also given.
Convergence analysis of Lawson's iteration for the polynomial and rational minimax approximations

Lei-Hong Zhang,Shanheng Han

2024-04-14

Abstract:Lawson's iteration is a classical and effective method for solving the linear (polynomial) minimax approximation in the complex plane. Extension of Lawson's iteration for the rational minimax approximation with both computationally high efficiency and theoretical guarantee is challenging. A recent work [L.-H. Zhang, L. Yang, W. H. Yang and Y.-N. Zhang, A convex dual programming for the rational minimax approximation and Lawson's iteration, 2023,

Numerical Analysis
On the convergence of polynomial approximation of rational functions

Guo-Jin Wang,Thomas W. Sederberg,Falai Chen

DOI: https://doi.org/10.1006/jath.1996.3060

IF: 0.993

1997-01-01

Journal of Approximation Theory

Abstract:This paper investigates the convergence condition :br the polynomial approximation of rational functions and rational curves. The main result, based on a hybrid expression of rational functions (or curves), is that two-point Hermite interpolation converges if all eigenvalue moduli of a certain r x r matrix are less than 2, where r is the degree of the rational function (or curve), and where the elements of the matrix are expressions involving only the denominator polynomial coefficients (weights) of the rational function (or curve). As a corollary for the special case of r = 1, a necessary and sufficient condition for convergence is also obtained which only involves the roots of the denominator of the rational function and which is shown to be superior to the condition obtained by rite traditional remainder theory for polynomial interpolation. For the low degree cases (r = 1, 2, and 3), concrete conditions are derived. Application to rational Bernstein-Bezier curves is discussed. (C) 1997 Academic Press.
On the finite section method for computing exponentials of doubly-infinite skew-Hermitian matrices

meiyue shao

DOI: https://doi.org/10.1016/j.laa.2014.03.021

IF: 1.307

2014-01-01

Linear Algebra and its Applications

Abstract:Computing the exponential of large-scale skew-Hermitian matrices or parts thereof is frequently required in applications. In this work, we consider the task of extracting finite diagonal blocks from a doubly-infinite skew-Hermitian matrix. These matrices usually have unbounded entries which impede the application of many classical techniques from approximation theory. We analyze the decay property of matrix exponentials for several classes of banded skew-Hermitian matrices. Then finite section methods based on the decay property are presented. We use several examples to demonstrate the effectiveness of these methods.
Composite orthogonal projection methods for large matrix eigenproblems

Jia Zhongxiao

DOI: https://doi.org/10.1007/BF02880075

1999-01-01

Abstract:For classical orthogonal projection methods for large matrix eigenproblems, it may be much more difficult for a Ritz vector to converge than for its corresponding Ritz value when the matrix in question is non-Hermitian. To this end, a class of new refined orthogonal projection methods has been proposed. It is proved that in some sense each refined method is a composite of two classical orthogonal projections, in which each refined approximate eigenvector is obtained by realizing a new one of some Hermitian semipositive definite matrix onto the same subspace. A priori error bounds on the refined approximate eigenvector are established in terms of the sine of acute angle of the normalized eigenvector and the subspace involved. It is shown that the sufficient conditions for convergence of the refined vector and that of the Ritz value are the same, so that the refined methods may be much more efficient than the classical ones.
Convergence analysis of Hermite approximations for analytic functions

Haiyong Wang,Lun Zhang

2023-12-13

Abstract:In this paper, we present a rigorous analysis of root-exponential convergence of Hermite approximations, including projection and interpolation methods, for functions that are analytic in an infinite strip containing the real axis and satisfy certain restrictions on the asymptotic behavior at infinity within this strip. Asymptotically sharp error bounds in the weighted and maximum norms are derived. The key ingredients of our analysis are some remarkable contour integral representations for the Hermite coefficients and the remainder of Hermite spectral interpolations. Further extensions to Gauss--Hermite quadrature, Hermite spectral differentiations, generalized Hermite spectral approximations and the scaling factor of Hermite approximation are also discussed. Numerical experiments confirm our theoretical results.

Numerical Analysis
A Projection Method Based on Extended Krylov Subspaces for Solving Sylvester Equations

Yiqin Lin,Bao Li,Yimin Wei

DOI: https://doi.org/10.5281/zenodo.1085555

2011-01-01

Abstract:—In this paper we study numerical methods for solving Sylvester matrix equations of the form AX + XB T + CD T = 0 . A new projection method is proposed. The union of Krylov subspaces in A and its inverse and the union of Krylov subspaces in B and its inverse are used as the right and left projection subspaces, respectively. The Arnoldi-like process for constructing the orthonormal basis of the projection subspaces is outlined. We show that the approximate solution is an exact solution of a perturbed Sylvester matrix equation. Moreover, exact expression for the norm of residual is derived and results on ﬁnite termination and convergence are presented. Some numerical examples are presented to illustrate the effectiveness of the proposed method.
A new projection method for solving large Sylvester equations

Liang Bao,Yiqin Lin,Yimin Wei

DOI: https://doi.org/10.1016/j.apnum.2006.07.005

IF: 2.994

2007-01-01

Applied Numerical Mathematics

Abstract:In this paper, we propose a new projection method based on global Arnoldi algorithm for solving large Sylvester matrix equations AX+XB+CD^T=0 and the large generalized Sylvester matrix equations of the form AXB+X+CD^T=0. We show how to extract low-rank approximate solutions to Sylvester matrix equations and generalized Sylvester matrix equations. Some theoretical results are given. Numerical tests report the effectiveness of these methods.
On convergence of a sketch-and-project method for the matrix equation

Wendi Bao,Zhiwei Guo,Weiguo Li,Ying Lv

DOI: https://doi.org/10.1007/s40314-024-02847-8

2024-07-13

Computational and Applied Mathematics

Abstract:In this paper, based on Lagrangian functions of the optimization problem we develop a sketch-and-project method for solving the linear matrix equation by introducing three parameters. A thorough convergence analysis on the proposed method is explored in details. A lower bound on the convergence rate and some convergence conditions are derived. By varying three parameters in the new method and convergence theorems, an array of well-known algorithms and their convergence results are recovered. Finally, numerical experiments are given to illustrate the effectiveness of recovered methods.

mathematics, applied
Linear Convergence Rates for Extrapolated Fixed Point Algorithms

Christian Bargetz,Victor I. Kolobov,Simeon Reich,Rafał Zalas

DOI: https://doi.org/10.48550/arXiv.1805.03932

2018-05-10

Abstract:We establish linear convergence rates for a certain class of extrapolated fixed point algorithms which are based on dynamic string-averaging methods in a real Hilbert space. This applies, in particular, to the extrapolated simultaneous and cyclic cutter methods. Our analysis covers the cases of both metric and subgradient projections.

Optimization and Control
Dense and Structured Matrix Computations —the Parallel QR Algorithm and Matrix Exponentials

Meiyue Shao

2013-01-01

Abstract:This thesis is concerned with two classical topics in matrix computations : The QR algorithm for solving nonsymmetric eigenvalue problems and the computation of matrix exponentials for two types of structured matrices. We focus on the performance in the former topic and on accuracy in the latter one. For computing all eigenvalues of a non-Hermitian matrix, the QR algorithm which iteratively computes a Schur decomposition of the matrix is the method of choice. We present a new parallel implementation of the multishift QR algorithm targeting distributed memory architectures. Starting from recent developments of the parallel multishift QR algorithm, we propose a number of algorithmic and implementation improvements. Guidelines concerning several important tunable algorithmic parameters are also provided. Numerous computational experiments confirm that our new implementation significantly outperforms previous parallel implementations of the QR algorithm. The computation of the exponential of a square matrix is also an important task in matrix computations. For a general dense matrix, the scaling and squaring method coupled with Pade approximation is the most popular approach. However, for an essentially nonnegative matrix (a real square matrix with nonnegative off-diagonal entries), truncated Taylor series rather than Pade approximation is preferred to achieve componentwise accuracy in the matrix exponential. We propose a method which efficiently computes all entries of the exponential of an essentially nonnegative matrix to high relative accuracy. Truncation and rounding error bounds, as well as numerical experiments demonstrate the efficiency and accuracy of our method. When the matrix is banded, the entries of its matrix exponential decay exponentially away from the main diagonal. We analyze the decay property for the exponentials of several classes of doubly-infinite skew-Hermitian matrices. Then finite section methods based on the decay property are established. We also propose a repeated doubling strategy which works well even when a priori error estimates are pessimistic or not easy to compute. Finally, numerical experiments are presented to illustrate the effectiveness of the finite section method.
Efficient scaling and squaring method for the matrix exponential

Sergio Blanes,Nikita Kopylov,Muaz Seydaoğlu

2024-04-19

Abstract:This work presents a new algorithm to compute the matrix exponential within a given tolerance. Combined with the scaling and squaring procedure, the algorithm incorporates Taylor, partitioned and classical Padé methods shown to be superior in performance to the approximants used in state-of-the-art software. The algorithm computes matrix--matrix products and also matrix inverses, but it can be implemented to avoid the computation of inverses, making it convenient for some problems. If the matrix A belongs to a Lie algebra, then exp(A) belongs to its associated Lie group, being a property which is preserved by diagonal Padé approximants, and the algorithm has another option to use only these. Numerical experiments show the superior performance with respect to state-of-the-art implementations.

Numerical Analysis
Summing divergent matrix series

Rongbiao Wang,JungHo Lee,Lek-Heng Lim

2024-05-30

Abstract:We extend several celebrated methods in classical analysis for summing series of complex numbers to series of complex matrices. These include the summation methods of Abel, Borel, Cesáro, Euler, Lambert, Nörlund, and Mittag-Leffler, which are frequently used to sum scalar series that are divergent in the conventional sense. One feature of our matrix extensions is that they are fully noncommutative generalizations of their scalar counterparts -- not only is the scalar series replaced by a matrix series, positive weights are replaced by positive definite matrix weights, order on $\mathbb{R}$ replaced by Loewner order, exponential function replaced by matrix exponential function, etc. We will establish the regularity of our matrix summation methods, i.e., when applied to a matrix series convergent in the conventional sense, we obtain the same value for the sum. Our second goal is to provide numerical algorithms that work in conjunction with these summation methods. We discuss how the block and mixed-block summation algorithms, the Kahan compensated summation algorithm, may be applied to matrix sums with similar roundoff error bounds. These summation methods and algorithms apply not only to power or Taylor series of matrices but to any general matrix series including matrix Fourier and Dirichlet series. We will demonstrate the utility of these summation methods: establishing a Fejér's theorem and alleviating the Gibbs phenomenon for matrix Fourier series; extending the domains of matrix functions and accurately evaluating them; enhancing the matrix Padé approximation and Schur--Parlett algorithms; and more.

Numerical Analysis,Classical Analysis and ODEs
A sketch-and-project method for solving the matrix equation AXB = C

Wendi Bao,Zhiwei Guo,Weiguo Li,Ying Lv,Jichao Wang

2023-06-06

Abstract:In this paper, based on an optimization problem, a sketch-and-project method for solving the linear matrix equation AXB = C is proposed. We provide a thorough convergence analysis for the new method and derive a lower bound on the convergence rate and some convergence conditions including the case that the coefficient matrix is rank deficient. By varying three parameters in the new method and convergence theorems, the new method recovers an array of well-known algorithms and their convergence results. Meanwhile, with the use of Gaussian sampling, we can obtain the Gaussian global randomized Kaczmarz (GaussGRK) method which shows some advantages in solving the matrix equation AXB = C. Finally, numerical experiments are given to illustrate the effectiveness of recovered methods.

Numerical Analysis
Adaptive Rational Krylov Methods for Exponential Runge–Kutta Integrators

Kai Bergermann,Martin Stoll

DOI: https://doi.org/10.1137/23m1559439

IF: 1.908

2024-03-06

SIAM Journal on Matrix Analysis and Applications

Abstract:SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 744-770, March 2024. We consider the solution of large stiff systems of ODEs with explicit exponential Runge–Kutta integrators. These problems arise from semidiscretized semilinear parabolic PDEs on continuous domains or on inherently discrete graph domains. A series of results reduces the requirement of computing linear combinations of [math]-functions in exponential integrators to the approximation of the action of a smaller number of matrix exponentials on certain vectors. State-of-the-art computational methods use polynomial Krylov subspaces of adaptive size for this task. They have the drawback that the required number of Krylov subspace iterations to obtain a desired tolerance increases drastically with the spectral radius of the discrete linear differential operator, e.g., the problem size. We present an approach that leverages rational Krylov subspace methods promising superior approximation qualities. We prove a novel a posteriori error estimate of rational Krylov approximations to the action of the matrix exponential on vectors for single time points, which allows for an adaptive approach similar to existing polynomial Krylov techniques. We discuss pole selection and the efficient solution of the arising sequences of shifted linear systems by direct and preconditioned iterative solvers. Numerical experiments show that our method outperforms the state of the art for sufficiently large spectral radii of the discrete linear differential operators. The key to this are approximately constant numbers of rational Krylov iterations, which enable a near-linear scaling of the runtime with respect to the problem size.

mathematics, applied
Adaptive rational Krylov methods for exponential Runge--Kutta integrators

Kai Bergermann,Martin Stoll

2023-08-23

Abstract:We consider the solution of large stiff systems of ordinary differential equations with explicit exponential Runge--Kutta integrators. These problems arise from semi-discretized semi-linear parabolic partial differential equations on continuous domains or on inherently discrete graph domains. A series of results reduces the requirement of computing linear combinations of $\varphi$-functions in exponential integrators to the approximation of the action of a smaller number of matrix exponentials on certain vectors. State-of-the-art computational methods use polynomial Krylov subspaces of adaptive size for this task. They have the drawback that the required number of Krylov subspace iterations to obtain a desired tolerance increase drastically with the spectral radius of the discrete linear differential operator, e.g., the problem size. We present an approach that leverages rational Krylov subspace methods promising superior approximation qualities. We prove a novel a-posteriori error estimate of rational Krylov approximations to the action of the matrix exponential on vectors for single time points, which allows for an adaptive approach similar to existing polynomial Krylov techniques. We discuss pole selection and the efficient solution of the arising sequences of shifted linear systems by direct and preconditioned iterative solvers. Numerical experiments show that our method outperforms the state of the art for sufficiently large spectral radii of the discrete linear differential operators. The key to this are approximately constant numbers of rational Krylov iterations, which enable a near-linear scaling of the runtime with respect to the problem size.

Numerical Analysis

On Analysis Of Projection Methods For Rational Function Approximation To The Matrix Exponential

Convergence Analysis of Projected Gradient Descent for Schatten-p Nonconvex Matrix Recovery

A Theoretical Comparison of Two Classes of Projection Methods

Convergence Rate Analysis of a Dykstra-Type Projection Algorithm

Convergence analysis of inexact proximal point algorithms on Hadamard manifolds

Convergence analysis of Lawson's iteration for the polynomial and rational minimax approximations

On the convergence of polynomial approximation of rational functions

On the finite section method for computing exponentials of doubly-infinite skew-Hermitian matrices

Composite orthogonal projection methods for large matrix eigenproblems

Convergence analysis of Hermite approximations for analytic functions

A Projection Method Based on Extended Krylov Subspaces for Solving Sylvester Equations

A new projection method for solving large Sylvester equations

On convergence of a sketch-and-project method for the matrix equation

Linear Convergence Rates for Extrapolated Fixed Point Algorithms

Dense and Structured Matrix Computations —the Parallel QR Algorithm and Matrix Exponentials

Efficient scaling and squaring method for the matrix exponential

Summing divergent matrix series

A sketch-and-project method for solving the matrix equation AXB = C

Adaptive Rational Krylov Methods for Exponential Runge–Kutta Integrators

Adaptive rational Krylov methods for exponential Runge--Kutta integrators