A New Method of Matrix Spectral Factorization

Gigla Janashia,Edem Lagvilava,Lasha Ephremidze

DOI: https://doi.org/10.48550/arXiv.0909.5361

2009-09-29

Abstract:A new method of matrix spectral factorization is proposed which reliably computes an approximate spectral factor of any matrix spectral density that admits spectral factorization

Complex Variables

What problem does this paper attempt to address?

On multivariable matrix spectral factorization method

Lasha Ephremidze,Ilya M. Spitkovsky

DOI: https://doi.org/10.1016/j.jmaa.2022.126300

IF: 1.417

2022-10-01

Journal of Mathematical Analysis and Applications

Abstract:Spectral factorization is a prominent tool with several important applications in various areas of applied science. Wiener and Masani proved the existence of matrix spectral factorization. Their theorem has been extended to the multivariable case by Helson and Lowdenslager. Solving the problem numerically is challenging in both situations, and also important due to its practical applications. Therefore, several authors have developed algorithms for factorization. The Janashia-Lagvilava algorithm is a relatively new method for matrix spectral factorization which has proved to be useful in several applications. In this paper, we extend this method to the multivariable case. Consequently, a new numerical algorithm for multivariable matrix spectral factorization is constructed.

mathematics, applied
A numerical algorithm for matrix spectral factorization on the real line

Lasha Ephremidze

2024-02-09

Abstract:In this paper, the Janashia-Lagvilava matrix spectral factorization algorithm, which is designed for power spectral density functions defined on the unit circle, is extended to the real line. The proposed algorithm can be used directly for continuous-time models

Complex Variables
An algorithm for J-spectral factorization of certain matrix functions

Lasha Ephremidze,Ilya Spitkovsky

DOI: https://doi.org/10.1109/cdc45484.2021.9683123

2021-12-14

Abstract:The problems of matrix spectral factorization and J-spectral factorization appear to be important for practical use in many MIMO control systems. We propose a numerical algorithm for J–spectral factorization which extends Janashia–Lagvilava matrix spectral factorization method to the indefinite case. The algorithm can be applied to matrices that have constant signatures for all leading principal submatrices. A numerical example is presented for illustrative purposes.
Spectral Factorization of Rank-Deficient Polynomial Matrix-Functions

Lasha Ephremidze,Edem Lagvilava

DOI: https://doi.org/10.48550/arXiv.1008.3122

2010-08-19

Abstract:A spectral factorization theorem is proved for polynomial rank-deficient matrix-functions. The theorem is used to construct paraunitary matrix-functions with first rows given.

Complex Variables
On the exact spectral factorization of rational matrix functions with applications to paraunitary filter banks

Lasha Ephremidze,Gennady Mishuris,Ilya Spitkovsky

2023-12-06

Abstract:In this paper, we enhance a recent algorithm for approximate spectral factorization of matrix functions, extending its capabilities to precisely factorize rational matrices when an exact lower-upper triangular factorization is available. This novel approach leverages a fundamental component of the improved algorithm for the precise design of rational paraunitary filter banks, allowing for the predetermined placement of zeros and poles. The introduced algorithm not only advances the state-of-the-art in spectral factorization but also opens new avenues for the tailored design of paraunitary filters with specific spectral properties, offering significant potential for applications in signal processing and beyond.

Complex Variables
Fast Spectral Low Rank Matrix Approximation

Haishan Ye,Zhihua Zhang

2015-01-01

Abstract:First, we extend the results of approximate matrix multiplication from the Frobenius norm to the spectral norm. Second, We develop a class of fast approximate generalized linear regression algorithms with respect to the spectral norm. Finally, We give a fast approximate SVD.
Binary Matrix Factorisation via Column Generation

Reka A. Kovacs,Oktay Gunluk,Raphael A. Hauser

DOI: https://doi.org/10.48550/arXiv.2011.04457

2021-08-04

Abstract:Identifying discrete patterns in binary data is an important dimensionality reduction tool in machine learning and data mining. In this paper, we consider the problem of low-rank binary matrix factorisation (BMF) under Boolean arithmetic. Due to the hardness of this problem, most previous attempts rely on heuristic techniques. We formulate the problem as a mixed integer linear program and use a large scale optimisation technique of column generation to solve it without the need of heuristic pattern mining. Our approach focuses on accuracy and on the provision of optimality guarantees. Experimental results on real world datasets demonstrate that our proposed method is effective at producing highly accurate factorisations and improves on the previously available best known results for 15 out of 24 problem instances.

Optimization and Control,Discrete Mathematics,Machine Learning
Spectral Methods for Matrix Product Factorization

Saieed Akbari,Yi-Zheng Fan,Fu-Tao Hu,Babak Miraftab,Yi Wang

2024-07-05

Abstract:A graph $G$ is factored into graphs $H$ and $K$ via a matrix product if there exist adjacency matrices $A$, $B$, and $C$ of $G$, $H$, and $K$, respectively, such that $A = BC$. In this paper, we study the spectral aspects of the matrix product of graphs, including regularity, bipartiteness, and connectivity. We show that if a graph $G$ is factored into a connected graph $H$ and a graph $K$ with no isolated vertices, then certain properties hold. If $H$ is non-bipartite, then $G$ is connected. If $H$ is bipartite and $G$ is not connected, then $K$ is a regular bipartite graph, and consequently, $n$ is even. Furthermore, we show that trees are not factorizable, which answers a question posed by Maghsoudi et al.

Combinatorics,Discrete Mathematics
Estimation of spectral gaps for sparse symmetric matrices

Michele Benzi,Michele Rinelli,Igor Simunec

2024-10-20

Abstract:In this paper we propose and analyze an algorithm for identifying spectral gaps of a real symmetric matrix $A$ by simultaneously approximating the traces of spectral projectors associated with multiple different spectral slices. Our method utilizes Hutchinson's stochastic trace estimator together with the Lanczos algorithm to approximate quadratic forms involving spectral projectors. Instead of focusing on determining the gap between two particular consecutive eigenvalues of $A$, we aim to find all gaps that are wider than a specified threshold. By examining the problem from this perspective, and thoroughly analyzing both the Hutchinson and the Lanczos components of the algorithm, we obtain error bounds that allow us to determine the numbers of Hutchinson's sample vectors and Lanczos iterations needed to ensure the detection of all gaps above the target width with high probability. In particular, we conclude that the most efficient strategy is to always use a single random sample vector for Hutchinson's estimator and concentrate all computational effort in the Lanczos algorithm. Our numerical experiments demonstrate the efficiency and reliability of this approach.

Numerical Analysis
Stochastic Subsampling for Factorizing Huge Matrices

Arthur Mensch,Julien Mairal,Bertrand Thirion,Gael Varoquaux

DOI: https://doi.org/10.1109/TSP.2017.2752697

2017-10-30

Abstract:We present a matrix-factorization algorithm that scales to input matrices with both huge number of rows and columns. Learned factors may be sparse or dense and/or non-negative, which makes our algorithm suitable for dictionary learning, sparse component analysis, and non-negative matrix factorization. Our algorithm streams matrix columns while subsampling them to iteratively learn the matrix factors. At each iteration, the row dimension of a new sample is reduced by subsampling, resulting in lower time complexity compared to a simple streaming algorithm. Our method comes with convergence guarantees to reach a stationary point of the matrix-factorization problem. We demonstrate its efficiency on massive functional Magnetic Resonance Imaging data (2 TB), and on patches extracted from hyperspectral images (103 GB). For both problems, which involve different penalties on rows and columns, we obtain significant speed-ups compared to state-of-the-art algorithms.

Machine Learning,Optimization and Control,Neurons and Cognition
Fifty Three Matrix Factorizations: A systematic approach

Alan Edelman,Sungwoo Jeong

DOI: https://doi.org/10.48550/arXiv.2104.08669

2022-02-15

Abstract:The success of matrix factorizations such as the singular value decomposition (SVD) has motivated the search for even more factorizations. We catalog 53 matrix factorizations, most of which we believe to be new. Our systematic approach, inspired by the generalized Cartan decomposition of Lie theory, also encompasses known factorizations such as the SVD, the symmetric eigendecomposition, the CS decomposition, the hyperbolic SVD, structured SVDs, the Takagi factorization, and others thereby covering familiar matrix factorizations as well as ones that were waiting to be discovered. We suggest that Lie theory has one way or another been lurking hidden in the foundations of the very successful field of matrix computations with applications routinely used in so many areas of computation. In this paper, we investigate consequences of the Cartan decomposition and the little known generalized Cartan decomposition for matrix factorizations. We believe that these factorizations once properly identified can lead to further work on algorithmic computations and applications.

Numerical Analysis
"Compress and eliminate" solver for symmetric positive definite sparse matrices

Daria A. Sushnikova,Ivan V. Oseledets

DOI: https://doi.org/10.48550/arXiv.1603.09133

2018-05-05

Abstract:We propose a new approximate factorization for solving linear systems with symmetric positive definite sparse matrices. In a nutshell the algorithm is to apply hierarchically block Gaussian elimination and additionally compress the fill-in. The systems that have efficient compression of the fill-in mostly arise from discretization of partial differential equations. We show that the resulting factorization can be used as an efficient preconditioner and compare the proposed approach with state-of-art direct and iterative solvers.

Numerical Analysis
Accelerated Structured Matrix Factorization

Lorenzo Schiavon,Bernardo Nipoti,Antonio Canale

DOI: https://doi.org/10.1080/10618600.2023.2301072

2024-02-09

Journal of Computational and Graphical Statistics

Abstract:Matrix factorization exploits the idea that, in complex high-dimensional data, the actual signal typically lies in lower-dimensional structures. These lower dimensional objects provide useful insights, with interpretation favored by sparse structures. Sparsity, in addition, is beneficial in terms of regularization and, thus, to avoid over-fitting. By exploiting Bayesian shrinkage priors, we devise a computationally convenient approach for high-dimensional matrix factorization. The dependence between row and column entities is modeled by inducing flexible sparse patterns within factors. The availability of external information is accounted for in such a way that structures are allowed while not imposed. Inspired by boosting algorithms, we pair the proposed approach with a numerical strategy relying on a sequential inclusion and estimation of low-rank contributions, with a data-driven stopping rule. Practical advantages of the proposed approach are demonstrated by means of a simulation study and the analysis of soccer heatmaps obtained from new generation tracking data. Supplementary materials for this article are available online.

statistics & probability
Iterative methods for solving factorized linear systems

Anna Ma,Deanna Needell,Aaditya Ramdas

DOI: https://doi.org/10.48550/arXiv.1701.07453

2019-01-10

Abstract:Stochastic iterative algorithms such as the Kaczmarz and Gauss-Seidel methods have gained recent attention because of their speed, simplicity, and the ability to approximately solve large-scale linear systems of equations without needing to access the entire matrix. In this work, we consider the setting where we wish to solve a linear system in a large matrix X that is stored in a factorized form, X = UV; this setting either arises naturally in many applications or may be imposed when working with large low-rank datasets for reasons of space required for storage. We propose a variant of the randomized Kaczmarz method for such systems that takes advantage of the factored form, and avoids computing X. We prove an exponential convergence rate and supplement our theoretical guarantees with experimental evidence demonstrating that the factored variant yields significant acceleration in convergence.

Numerical Analysis
Approximating Spectral Densities of Large Matrices

Lin,Yousef Saad,Chao Yang

DOI: https://doi.org/10.1137/130934283

IF: 10.2

2013-01-01

SIAM Review

Abstract:In physics, it is sometimes desirable to compute the so-called density of states (DOS), also known as the spectral density, of a real symmetric matrix A. The spectral density can be viewed as a probability density distribution that measures the likelihood of finding eigenvalues near some point on the real line. The most straightforward way to obtain this density is to compute all eigenvalues of A, but this approach is generally costly and wasteful, especially for matrices of large dimension. There exist alternative methods that allow us to estimate the spectral density function at much lower cost. The major computational cost of these methods is in multiplying A with a number of vectors, which makes them appealing for large-scale problems where products of the matrix A with arbitrary vectors are relatively inexpensive. This article defines the problem of estimating the spectral density carefully and discusses how to measure the accuracy of an approximate spectral density. It then surveys a few known methods for estimating the spectral density and considers variations of existing methods. All methods are discussed from a numerical linear algebra point of view.
Improved Spectral Density Estimation via Explicit and Implicit Deflation

Rajarshi Bhattacharjee,Rajesh Jayaram,Cameron Musco,Christopher Musco,Archan Ray

2024-10-29

Abstract:We study algorithms for approximating the spectral density of a symmetric matrix $A$ that is accessed through matrix-vector product queries. By combining a previously studied Chebyshev polynomial moment matching method with a deflation step that approximately projects off the largest magnitude eigendirections of $A$ before estimating the spectral density, we give an $\epsilon\cdot\sigma_\ell(A)$ error approximation to the spectral density in the Wasserstein-$1$ metric using $O(\ell\log n+ 1/\epsilon)$ matrix-vector products, where $\sigma_\ell(A)$ is the $\ell^{th}$ largest singular value of $A$. In the common case when $A$ exhibits fast singular value decay, our bound can be much stronger than prior work, which gives an error bound of $\epsilon \cdot ||A||_2$ using $O(1/\epsilon)$ matrix-vector products. We also show that it is nearly tight: any algorithm giving error $\epsilon \cdot \sigma_\ell(A)$ must use $\Omega(\ell+1/\epsilon)$ matrix-vector products. We further show that the popular Stochastic Lanczos Quadrature (SLQ) method matches the above bound, even though SLQ itself is parameter-free and performs no explicit deflation. This bound explains the strong practical performance of SLQ, and motivates a simple variant of SLQ that achieves an even tighter error bound. Our error bound for SLQ leverages an analysis that views it as an implicit polynomial moment matching method, along with recent results on low-rank approximation with single-vector Krylov methods. We use these results to show that the method can perform implicit deflation as part of moment matching.

Data Structures and Algorithms,Numerical Analysis
An effective criterion for a stable factorisation of strictly non-singular 2 × 2 matrix functions: use of the ExactMPF package

N. V. Adukova,V. M. Adukov,G. Mishuris

DOI: https://doi.org/10.1098/rspa.2024.0116

2024-10-11

Proceedings of the Royal Society A Mathematical Physical and Engineering Sciences

Abstract:In this paper, we propose a method to factorise arbitrary strictly non-singular 2 × 2 matrix functions, allowing for stable factorisation. For this purpose, we use the E x a c t M P F package working within the Maple environment previously developed by the authors and perform an exact factorisation of a non-singular polynomial matrix function. A crucial point in the present analysis is the evaluation of a stability region of the canonical factorisation of the polynomial matrix functions. This, in turn, allows us to propose a sufficient condition for the given matrix function admitting stable factorisation.

multidisciplinary sciences
An effective criterion for a stable factorisation of strictly nonsingular 2x2 matrix functions. Utilisation of the ExactMPF package

Natalia Adukova,Victor Adukov,Gennady Mishuris

2024-06-12

Abstract:In this paper, we propose a method to factorise of arbitrary strictly nonsingular 2x2 matrix functions allowing for stable factorisation. For this purpose, we utilise the ExactMPF package working within the Maple environment previously developed by the authors and performing an exact factorisation of a nonsingular polynomial matrix function. A crucial point in the present analysis is the evaluation of a stability region of the canonical factorisation of the polynomial matrix functions. This, in turn, allows us to propose a sufficient condition for the given matrix function admitting stable factorisation.

Rings and Algebras
Spectral Condition-Number Estimation of Large Sparse Matrices

Haim Avron,Alex Druinsky,Sivan Toledo

DOI: https://doi.org/10.48550/arXiv.1301.1107

2018-08-30

Abstract:We describe a randomized Krylov-subspace method for estimating the spectral condition number of a real matrix A or indicating that it is numerically rank deficient. The main difficulty in estimating the condition number is the estimation of the smallest singular value \sigma_{\min} of A. Our method estimates this value by solving a consistent linear least-squares problem with a known solution using a specific Krylov-subspace method called LSQR. In this method, the forward error tends to concentrate in the direction of a right singular vector corresponding to \sigma_{\min}. Extensive experiments show that the method is able to estimate well the condition number of a wide array of matrices. It can sometimes estimate the condition number when running a dense SVD would be impractical due to the computational cost or the memory requirements. The method uses very little memory (it inherits this property from LSQR) and it works equally well on square and rectangular matrices.

Numerical Analysis
A Fast Spectral Relaxation Approach to Matrix Completion Via Kronecker Products.

Hui Zhao,Jiuqiang Han,Naiyan Wang,Congfu Xu,Zhihua Zhang

DOI: https://doi.org/10.1609/aaai.v25i1.7913

2011-01-01

Abstract:In the existing methods for solving matrix completion, such as singular value thresholding (SVT), soft-impute and ﬁxed point continuation (FPCA) algorithms, it is typically required to repeatedly implement singular value decompositions (SVD) of matrices.When the size of the matrix in question is large, the computational complexity of ﬁnding a solution is costly. To reduce this expensive computational complexity, we apply Kronecker products to handle the matrix completion problem. In particular, we propose using Kronecker factorization, which approximates a matrix by the Kronecker product of several matrices of smaller sizes. Weintroduce Kronecker factorization into the soft-impute framework and devise an effective matrix completion algorithm.Especially when the factorized matrices have about the samesizes, the computational complexity of our algorithm is improved substantially.

A New Method of Matrix Spectral Factorization

On multivariable matrix spectral factorization method

A numerical algorithm for matrix spectral factorization on the real line

An algorithm for J-spectral factorization of certain matrix functions

Spectral Factorization of Rank-Deficient Polynomial Matrix-Functions

On the exact spectral factorization of rational matrix functions with applications to paraunitary filter banks

Fast Spectral Low Rank Matrix Approximation

Binary Matrix Factorisation via Column Generation

Spectral Methods for Matrix Product Factorization

Estimation of spectral gaps for sparse symmetric matrices

Stochastic Subsampling for Factorizing Huge Matrices

Fifty Three Matrix Factorizations: A systematic approach

"Compress and eliminate" solver for symmetric positive definite sparse matrices

Accelerated Structured Matrix Factorization

Iterative methods for solving factorized linear systems

Approximating Spectral Densities of Large Matrices

Improved Spectral Density Estimation via Explicit and Implicit Deflation

An effective criterion for a stable factorisation of strictly non-singular 2 × 2 matrix functions: use of the ExactMPF package

An effective criterion for a stable factorisation of strictly nonsingular 2x2 matrix functions. Utilisation of the ExactMPF package

Spectral Condition-Number Estimation of Large Sparse Matrices

A Fast Spectral Relaxation Approach to Matrix Completion Via Kronecker Products.