A Fast Solver for Tridiagonal Toeplitz Systems with Multiple Right-Hand Sides

Shahin Hasanbeigi

2024-06-11

Abstract:In this work, we introduce a novel approach for solving tridiagonal Toeplitz systems with multiple right-hand sides.

Numerical Analysis

What problem does this paper attempt to address?

A fast Solver for Pentadiagonal Toeplitz Systems

Shahin Hasanbeigi

2024-05-09

Abstract:The objective of this work is to present a novel approach for the solution of Pentadiagonal Toeplitz systems of equations that is both faster and more effective than existing classical direct methods. The distinctive structure of Pentadiagonal Toeplitz matrices can be leveraged to devise an algorithm for solving upper triangle systems, rather than the original system. This approach is considerably more straightforward and expeditious than classical methods such as LU and Gauss Eliminations. A comparison with the LU and PLU methods demonstrates the efficacy of our novel algorithm. Furthermore, numerical tests substantiate this efficacy.

Numerical Analysis
Numerical algorithms for the fast and reliable solution of periodic tridiagonal Toeplitz linear systems

Ji-Teng Jia,Yi-Fan Wang

DOI: https://doi.org/10.1007/s11075-024-01795-y

IF: 2.37

2024-03-22

Numerical Algorithms

Abstract:In the present paper, we consider the solution of periodic tridiagonal Toeplitz linear systems. By exploiting the special low-rank and Toeplitz structure, we give a new structure-preserving decomposition form of the coefficient matrix. Based on this matrix decomposition form and combined with elementary column operations, we give different algorithms for the fast and reliable solutions of three types of periodic tridiagonal Toeplitz linear systems. Compared with the decomposition method, our proposed algorithms take less computational costs and need less memory storage and data transmission. Numerical experiments are given to demonstrate the accuracy and efficiency of our algorithms. All of the experiments are performed on a computer with the aid of programs written in MATLAB.

mathematics, applied
An efficient numerical algorithm for solving linear systems with cyclic tridiagonal coefficient matrices

Jia, Ji-Teng,Xie, Rong,Wang, Yi-Fan

DOI: https://doi.org/10.1007/s10910-024-01631-7

2024-06-09

Journal of Mathematical Chemistry

Abstract:In the present paper, we mainly consider the direct solution of cyclic tridiagonal linear systems. By using the specific low-rank and Toeplitz-like structure, we derive a structure-preserving factorization of the coefficient matrix. Based on the combination of such matrix factorization and Sherman–Morrison–Woodbury formula, we then propose a cost-efficient algorithm for numerically solving cyclic tridiagonal linear systems, which requires less memory storage and data transmission. Furthermore, we show that the structure-preserving matrix factorization can provide us with an explicit formula for n -th order cyclic tridiagonal determinants. Numerical examples are given to demonstrate the performance and efficiency of our algorithm. All of the experiments are performed on a computer with the aid of programs written in MATLAB.

chemistry, multidisciplinary,mathematics, interdisciplinary applications
GPU-based Parallel Solver Via the Kantorovich Theorem for the Nonlinear Bernstein Polynomial Systems.

Feifei Wei,Jieqing Feng,Hongwei Lin

DOI: https://doi.org/10.1016/j.camwa.2011.05.058

IF: 3.218

2011-01-01

Computers & Mathematics with Applications

Abstract:This paper proposes a parallel solver for the nonlinear systems in Bernstein form based on subdivision and the Newton–Raphson method, where the Kantorovich theorem is employed to identify the existence of a unique root and guarantee the convergence of the Newton–Raphson iterations. Since the Kantorovich theorem accommodates a singular Jacobian at the root, the proposed algorithm performs well in a multiple root case. Moreover, the solver is designed and implemented in parallel on Graphics Processing Unit(GPU) with SIMD architecture; thus, efficiency for solving a large number of systems is improved greatly, an observation validated by our experimental results.
Fast algorithms for the solution of perturbed symmetric Toeplitz linear system and its applications

Xing Zhang,Yanpeng Zheng,Zhaolin Jiang

DOI: https://doi.org/10.1007/s40314-024-02773-9

2024-05-24

Computational and Applied Mathematics

Abstract:We consider solving the perturbed symmetric Toeplitz linear system and the multiplication of the perturbed symmetric Toeplitz matrix and the vector. In terms of solving the perturbed symmetric Toeplitz linear system, we present two new approaches for the solution. Based on the decomposing, embedding and trigonometric transformation methods for symmetric Toeplitz matrix–vector multiplication, and order-reduction algorithms, five novel methods for the product of the perturbed symmetric Toeplitz matrix and the vector are given. The proposed fast algorithms are applied to image encryption and decryption and numerical results are reported to show the proposed algorithms are promising.

mathematics, applied
A Parallel Iterative Method For Solving Symmetrical Tridiagonal Extreme Eigenpair Problems

Zhenyue Zhang

DOI: https://doi.org/10.1016/0898-1221(93)90057-3

IF: 3.218

1993-01-01

Computers & Mathematics with Applications

Abstract:In this paper, we propose a parallel iterative method for calculating the extreme eigenpair (the largest or smallest eigenvalue and corresponding eigenvector) of a large symmetric tridiagonal matrix. It is based upon a divide and repeated, rank-one modification technique. The rank-one modification with a parameter only changes one diagonal element of each submatrix. We present a basic theory for subdividing the extremal eigenpair problem and then prove several convergence theorems that show the convergence of the iteration scheme for any positive initial modification parameter and the asymptotical quadratic convergence rate. Some numerical experiments are given, which show the efficiency of the parallel algorithm.
A parallel iterative method for solving symmetric tridiagonal extreme eigenpair problems

Zhenyue Zhang

DOI: https://doi.org/10.1016/0898-1221(93)90057-3

IF: 3.218

1993-01-01

Computers & Mathematics with Applications

Abstract:In this paper, we propose a parallel iterative method for calculating the extreme eigenpair (the largest or smallest eigenvalue and corresponding eigenvector) of a large symmetric tridiagonal matrix. It is based upon a divide and repeated, rank-one modification technique. The rank-one modification with a parameter only changes one diagonal element of each submatrix. We present a basic theory for subdividing the extremal eigenpair problem and then prove several convergence theorems that show the convergence of the iteration scheme for any positive initial modification parameter and the asymptotical quadratic convergence rate. Some numerical experiments are given, which show the efficiency of the parallel algorithm.
Superfast Tikhonov Regularization of Toeplitz Systems

Christopher Turnes,Doru Balcan,Justin Romberg

DOI: https://doi.org/10.1109/TSP.2014.2330800

2013-02-28

Abstract:Toeplitz-structured linear systems arise often in practical engineering problems. Correspondingly, a number of algorithms have been developed that exploit Toeplitz structure to gain computational efficiency when solving these systems. The earliest "fast" algorithms for Toeplitz systems required O(n^2) operations, while more recent "superfast" algorithms reduce the cost to O(n (log n)^2) or below. In this work, we present a superfast algorithm for Tikhonov regularization of Toeplitz systems. Using an "extension-and-transformation" technique, our algorithm translates a Tikhonov-regularized Toeplitz system into a type of specialized polynomial problem known as tangential interpolation. Under this formulation, we can compute the solution in only O(n (log n)^2) operations. We use numerical simulations to demonstrate our algorithm's complexity and verify that it returns stable solutions.

Numerical Analysis
Direct Solution Method for System of Linear Equations

Anoosh Abdy

DOI: https://doi.org/10.48550/arXiv.1104.5639

2011-04-29

Abstract:Direct solution of simultaneous linear equations is regarded to be slow for large systems of equations and requires special treatment to avoid numerical instability. A new method is proposed that addresses the numerical instability without any special treatments. The method uses the orthogonal matrices to achieve numerical stability.

Numerical Analysis
Explicit inverse of symmetric, tridiagonal near Toeplitz matrices Part II: with weakly diagonally dominant Toeplitz

Bakytzhan Kurmanbek,Yogi Erlangga,Yerlan Amanbek

2024-07-10

Abstract:In this paper, we provide explicit formulas for the exact inverses of the symmetric tridiagonal near-Toeplitz matrices characterized by weak diagonal dominance in the Toeplitz part. Furthermore, these findings extend to scenarios where the corners of the near Toeplitz matrices lack diagonal dominance ($|\widetilde{b}| < 1$). Additionally, we compute the row sums and traces of the inverse matrices, thereby deriving upper bounds for their infinite norms. To demonstrate the practical applicability of our theoretical results, we present numerical examples addressing numerical solution of the Fisher problem using the fixed point method. Our findings reveal that the convergence rates of fixed-point iterations closely align with the expected rates, and there is minimal disparity between the upper bounds and the infinite norm of the inverse matrix. Specifically, this observation holds true for $|b| = 2$ with $|\widetilde{b}| \geq 1$. In other cases, there exists potential to enhance the obtained upper bounds.

Numerical Analysis
An Inverse Extreme Eigenpair Problem and Its Parallel Iterative Solution for Symmetric Tridiagonal Matrices

ZY ZHANG

DOI: https://doi.org/10.1137/0732072

IF: 3.039

1995-01-01

SIAM Journal on Numerical Analysis

Abstract:This article concerns computing the largest (or smallest) eigenvalue and corresponding eigenvector of a large irreducible symmetric tridiagonal matrix. We divide the original extreme eigenpair problem into p subproblems when the matrix is partitioned in p-block form. An inverse extreme eigenpair problem associated with the divide method is considered: Find a (p - 1)-dimensional vector a such that the largest eigenvalue lambda of a given symmetric tridiagonal matrix T is the common largest eigenvalues of p small matrices T-i(a) which are updated matrices of the diagonal blocks T-i of T with rank-one/two corrections. We prove that the inverse problem is solvable and has a unique positive solution and that the corresponding eigenvector of T-i(a) with the positive solution a forms a section of the eigenvector of T corresponding to its largest eigenvalue. An algorithm is proposed for solving the inverse problem iteratively, on which a system of nonlinear equations should be solved for each of its iterative steps. It is proved that the iterative scheme converges for any positive initial vector and that its convergence is asymptotically quadratic. The new method can be used to compute the extreme eigenpairs of T and is well suited for parallel implementation. Some numerical tests were performed, which show the efficiency of the new parallelizable algorithm.
Spectral characterizations and integer powers of tridiagonal 2-Toeplitz matrices

Maryam Shams Solary,Stefano Serra-Capizzano

DOI: https://doi.org/10.1007/s11075-024-01863-3

IF: 2.37

2024-06-23

Numerical Algorithms

Abstract:In this note, we consider real nonsymmetric tridiagonal 2-Toeplitz matrices . First we give the asymptotic spectral and singular value distribution of the whole matrix-sequence , which is described via two eigenvalue functions of a matrix-valued symbol. In connection with the above findings, we provide a characterization of the eigenvalues and eigenvectors of real tridiagonal 2-Toeplitz matrices of even order, that can be turned into a numerical effective scheme for the computation of all the entries of , n even and l positive and small compared to n . We recall that a corresponding eigenvalue decomposition for odd order tridiagonal 2-Toeplitz matrices was found previously, while, for even orders, an implicit formula for all the eigenvalues is obtained.

mathematics, applied
On the invertibility of matrices with a double saddle-point structure

Fatemeh P. A. Beik,Chen Greif,Manfred Trummer

2024-07-03

Abstract:We establish necessary and sufficient conditions for invertibility of symmetric three-by-three block matrices having a double saddle-point structure \fb{that guarantee the unique solvability of double saddle-point systems}. We consider various scenarios, including the case where all diagonal blocks are allowed to be rank deficient. Under certain conditions related to the nullity of the blocks and intersections of their kernels, an explicit formula for the inverse is derived.

Numerical Analysis
Combinatorial and Recurrent Approaches for Efficient Matrix Inversion: Sub-cubic algorithms leveraging Fast Matrix products

Mohamed Kamel Riahi

2023-07-15

Abstract:In this paper, we introduce novel fast matrix inversion algorithms that leverage triangular decomposition and recurrent formalism, incorporating Strassen's fast matrix multiplication. Our research places particular emphasis on triangular matrices, where we propose a novel computational approach based on combinatorial techniques for finding the inverse of a general non-singular triangular matrix. Unlike iterative methods, our combinatorial approach for (block) triangular-type matrices enables direct computation of the matrix inverse through a nonlinear combination of carefully selected combinatorial entries from the initial matrix. This unique characteristic makes our proposed method fully parallelizable, offering significant potential for efficient implementation on parallel computing architectures. Our approach demonstrates intriguing features that allow the derivation of recurrent relations for constructing the matrix inverse. By combining the (block) combinatorial approach, with a recursive triangular split method for inverting triangular matrices, we develop potentially competitive algorithms that strike a balance between efficiency and accuracy. We provide rigorous mathematical proofs of the newly presented method. Additionally, we conduct extensive numerical tests to showcase its applicability and efficiency. The comprehensive evaluation and experimental results presented in this paper confirm the practical utility of our proposed algorithms, demonstrating their superiority over classical approaches in terms of computational efficiency.

Numerical Analysis,Combinatorics
A Novel Compute-Efficient Tridiagonal Solver for Many-Core Architectures

Kan Liu,Wei Xue

DOI: https://doi.org/10.1109/tpds.2022.3214762

IF: 5.3

2022-01-01

IEEE Transactions on Parallel and Distributed Systems

Abstract:The tridiagonal solver is an important kernel and is widely supported in mainstream numerical libraries. While parallel algorithms have been studied for many-core architectures, the performance of current algorithms and implementations is still hindered by input size sensitivity and cross-platform portability. In this paper, we propose a novel algorithm WM-pGE for the batched solution of diagonally dominant tridiagonal systems. The algorithm balances the key design objectives, including computation complexity, memory complexity, parallelism, and input size sensitivity, better than existing algorithms. Moreover, an elegant formulation is presented to show the implementation and cross-platform optimization without loss of efficiency and generality, by extracting the platform-dependent works into only four vector operators. The results from our batched tridiagonal experiments show that the proposed algorithm outperforms the prior work PCR-pThomas by 25% and 12% on NVIDIA Tesla V100 in single and double precision, respectively. On Intel KNL, our method achieves a 10% improvement in performance over PCR-pThomas in double precision.
A Fast Tridiagonal Solver for Intel MIC Architecture

Xinliang Wang,Wei Xue,Jidong Zhai,Yangtong Xu,Weimin Zheng,Hai Xiang Lin

DOI: https://doi.org/10.1109/IPDPS.2016.83

2016-01-01

Abstract:The tridiagonal solver is an important kernel in many scientific and engineering applications. Although quite a few parallel algorithms have been exploited recently, challenges still remain when solving tridiagonal systems on many-core architectures. In this paper, quantitative analysis is conducted to guide the selection of algorithms on different architectures, and a fast register-PCR-pThomas algorithm for Intel MIC architecture is presented to achieve the best utilization of in-core vectorization and registers. By choosing the most suitable number of PCR reductions, we further propose an improved algorithm, named register-PCR-half-pThomas algorithm, which minimizes the computational cost and the number of registers for use. The optimizations of manual prefetching and strength reduction are also discussed for the new algorithm. Evaluation on Intel Xeon Phi 7110P shows that our register-PCR-pThomas can outperform the MKL solver by 7.7 times in single precision and 3.4 times in double precision. Moreover, our optimized register-PCR-half-pThomas can further boost the performance by another 43.9% in single precision and 20.4% in double precision. Our best algorithm can outperform the latest official GPU library (cuSPARSE) on Nvidia K40 by 3.7 times and 2.4 times in single and double precision, respectively.
Deflated Iterative Methods for Linear Equations with Multiple Right-Hand Sides

Ronald B. Morgan,Walter Wilcox

DOI: https://doi.org/10.48550/arXiv.0707.0505

2007-07-03

Mathematical Physics

Abstract:A new approach is discussed for solving large nonsymmetric systems of linear equations with multiple right-hand sides. The first system is solved with a deflated GMRES method that generates eigenvector information at the same time that the linear equations are solved. Subsequent systems are solved by combining restarted GMRES with a projection over the previously determined eigenvectors. This approach offers an alternative to block methods, and it can also be combined with a block method. It is useful when there are a limited number of small eigenvalues that slow the convergence. An example is given showing significant improvement for a problem from quantum chromodynamics. The second and subsequent right-hand sides are solved much quicker than without the deflation. This new approach is relatively simple to implement and is very efficient compared to other deflation methods.
Tridiagonal and single-pair matrices and the inverse sum of two single-pair matrices

Sebastien Bossu

2024-02-29

Abstract:A novel factorization for the sum of two single-pair matrices is established as product of lower-triangular, tridiagonal, and upper-triangular matrices, leading to semi-closed-form formulas for tridiagonal matrix inversion. Subsequent factorizations are established, leading to semi-closed-form formulas for the inverse sum of two single-pair matrices. An application to derive the symbolic inverse of a particular Gram matrix is presented, and the numerical stability of the formulas is studied.

Rings and Algebras
A two-step algorithm for solving singular linear systems with index one.

Jieyong Zhou,Yimin Wei

DOI: https://doi.org/10.1016/j.amc.2005.07.027

IF: 4.397

2006-01-01

Applied Mathematics and Computation

Abstract:In this paper, a two-step algorithm for solving singular linear systems is presented. We compare this algorithm with a DGCR type algorithm [A. Sidi, A unified approach to Krylov subspace methods for the Drazin-inverse solution of singular non-symmetric linear systems, Linear Algebra Appl. 298 (1999) 99–113] by numerical experiments. An error analysis is given.
A Hybrid Direct-Iterative Method for Solving KKT Linear Systems

Shaked Regev,Nai-Yuan Chiang,Eric Darve,Cosmin G. Petra,Michael A. Saunders,Kasia Świrydowicz,Slaven Peleš

DOI: https://doi.org/10.48550/arXiv.2110.03636

2021-10-08

Abstract:We propose a solution strategy for linear systems arising in interior method optimization, which is suitable for implementation on hardware accelerators such as graphical processing units (GPUs). The current gold standard for solving these systems is the LDL^T factorization. However, LDL^T requires pivoting during factorization, which substantially increases communication cost and degrades performance on GPUs. Our novel approach solves a large indefinite system by solving multiple smaller positive definite systems, using an iterative solve for the Schur complement and an inner direct solve (via Cholesky factorization) within each iteration. Cholesky is stable without pivoting, thereby reducing communication and allowing reuse of the symbolic factorization. We demonstrate the practicality of our approach and show that on large systems it can efficiently utilize GPUs and outperform LDL^T factorization of the full system.

Optimization and Control,Distributed, Parallel, and Cluster Computing

A Fast Solver for Tridiagonal Toeplitz Systems with Multiple Right-Hand Sides

A fast Solver for Pentadiagonal Toeplitz Systems

Numerical algorithms for the fast and reliable solution of periodic tridiagonal Toeplitz linear systems

An efficient numerical algorithm for solving linear systems with cyclic tridiagonal coefficient matrices

GPU-based Parallel Solver Via the Kantorovich Theorem for the Nonlinear Bernstein Polynomial Systems.

Fast algorithms for the solution of perturbed symmetric Toeplitz linear system and its applications

A Parallel Iterative Method For Solving Symmetrical Tridiagonal Extreme Eigenpair Problems

A parallel iterative method for solving symmetric tridiagonal extreme eigenpair problems

Superfast Tikhonov Regularization of Toeplitz Systems

Direct Solution Method for System of Linear Equations

Explicit inverse of symmetric, tridiagonal near Toeplitz matrices Part II: with weakly diagonally dominant Toeplitz

An Inverse Extreme Eigenpair Problem and Its Parallel Iterative Solution for Symmetric Tridiagonal Matrices

Spectral characterizations and integer powers of tridiagonal 2-Toeplitz matrices

On the invertibility of matrices with a double saddle-point structure

Combinatorial and Recurrent Approaches for Efficient Matrix Inversion: Sub-cubic algorithms leveraging Fast Matrix products

A Novel Compute-Efficient Tridiagonal Solver for Many-Core Architectures

A Fast Tridiagonal Solver for Intel MIC Architecture

Deflated Iterative Methods for Linear Equations with Multiple Right-Hand Sides

Tridiagonal and single-pair matrices and the inverse sum of two single-pair matrices

A two-step algorithm for solving singular linear systems with index one.

A Hybrid Direct-Iterative Method for Solving KKT Linear Systems