Generalized Singular Value Decompositions: A proposal for a standardized nomenclature

Bart LR De Moor, Gene H Golub

1989-01-01

Abstract:

What problem does this paper attempt to address?

A Conditional Singular Value Decomposition

Qi Deng

2024-02-26

Abstract:We propose a Conditional Singular Value Decomposition in the form of A_{mn} =H_{mk} B_{kl} M_{ln}^* for given general matrices A_{mn} and B_{kl} .

General Mathematics
A Higher-Order Generalized Singular Value Decomposition for Rank Deficient Matrices

Idris Kempf,Paul J. Goulart,Stephen R. Duncan

DOI: https://doi.org/10.48550/arXiv.2102.09822

2022-06-20

Abstract:The higher-order generalized singular value decomposition (HO-GSVD) is a matrix factorization technique that extends the GSVD to $N \ge 2$ data matrices, and can be used to identify shared subspaces in multiple large-scale datasets with different row dimensions. The standard HO-GSVD factors $N$ matrices $A_i\in\mathbb{R}^{m_i\times n}$ as $A_i=U_i\Sigma_i V^\text{T}$, but requires that each of the matrices $A_i$ has full column rank. We propose a modification of the HO-GSVD that extends its applicability to rank-deficient data matrices $A_i$. If the matrix of stacked $A_i$ has full rank, we show that the properties of the original HO-GSVD extend to our approach. We extend the notion of common subspaces to isolated subspaces, which identify features that are unique to one $A_i$. We also extend our results to the higher-order cosine-sine decomposition (HO-CSD), which is closely related to the HO-GSVD. Our extension of the standard HO-GSVD allows its application to datasets with with $m_i<n$ or $\text{rank}(A_i)<n$, such as are encountered in bioinformatics, neuroscience, control theory or classification problems.

Numerical Analysis
Singular value decomposition of complexes

Danielle A. Brake,Jonathan D. Hauenstein,Frank-Olaf Schreyer,Andrew J. Sommese,Michael E. Stillman

DOI: https://doi.org/10.48550/arXiv.1804.09838

2018-05-30

Abstract:Singular value decompositions of matrices are widely used in numerical linear algebra with many applications. In this paper, we extend the notion of singular value decompositions to finite complexes of real vector spaces. We provide two methods to compute them and present several applications.

Numerical Analysis
High Order Numerical Methods To Approximate The Singular Value Decomposition

Diego Armentano,Jean-Claude Yakoubsohn

2023-09-12

Abstract:In this paper, we present a class of high order methods to approximate the singular value decomposition of a given complex matrix (SVD). To the best of our knowledge, only methods up to order three appear in the the literature. A first part is dedicated to defline and analyse this class of method in the regular case, i.e., when the singular values are pairwise distinct. The construction is based on a perturbation analysis of a suitable system of associated to the SVD (SVD system). More precisely, for an integer $p$ be given, we define a sequence which converges with an order $p + 1$ towards the left-right singular vectors and the singular values if the initial approximation of the SVD system satisfies a condition which depends on three quantities : the norm of initial approximation of the SVD system, the greatest singular value and the greatest inverse of the modulus of the difference between the singular values. From a numerical computational point of view, this furnishes a very efficient simple test to prove and certifiy the existence of a SVD in neighborhood of the initial approximation. We generalize these result in the case of clusters of singular values. We show also how to use the result of regular case to detect the clusters of singular values and to define a notion of deflation of the SVD. Moreover numerical experiments confirm the theoretical results.

Numerical Analysis
Randomized dual singular value decomposition

Mengyu Wang,Jingchun Zhou,Hanyu Li

2024-07-24

Abstract:We first propose a concise singular value decomposition of dual matrices. Then, the randomized version of the decomposition is presented. It can significantly reduce the computational cost while maintaining the similar accuracy. We analyze the theoretical properties and illuminate the numerical performance of the randomized algorithm.

Numerical Analysis
The Singular Value Decomposition, Applications and Beyond.

Zhihua Zhang

DOI: https://doi.org/10.48550/arxiv.1510.08532

2015-01-01

Abstract:The singular value decomposition (SVD) is not only a classical theory in matrix computation and analysis, but also is a powerful tool in machine learning and modern data analysis. In this tutorial we first study the basic notion of SVD and then show the central role of SVD in matrices. Using majorization theory, we consider variational principles of singular values and eigenvalues. Built on SVD and a theory of symmetric gauge functions, we discuss unitarily invariant norms, which are then used to formulate general results for matrix low rank approximation. We study the subdifferentials of unitarily invariant norms. These results would be potentially useful in many machine learning problems such as matrix completion and matrix data classification. Finally, we discuss matrix low rank approximation and its recent developments such as randomized SVD, approximate matrix multiplication, CUR decomposition, and Nystrom approximation. Randomized algorithms are important approaches to large scale SVD as well as fast matrix computations.
Singular Value Decomposition Approximation via Kronecker Summations for Imaging Applications

Clarissa Garvey,Chang Meng,James G. Nagy

DOI: https://doi.org/10.48550/arXiv.1803.11525

2018-04-02

Abstract:In this paper we propose an approach to approximate a truncated singular value decomposition of a large structured matrix. By first decomposing the matrix into a sum of Kronecker products, our approach can be used to approximate a large number of singular values and vectors more efficiently than other well known schemes, such as randomized matrix algorithms or iterative algorithms based on Golub-Kahan bidiagonalization. We provide theoretical results and numerical experiments to demonstrate the accuracy of our approximation and show how the approximation can be used to solve large scale ill-posed inverse problems, either as an approximate filtering method, or as a preconditioner to accelerate iterative algorithms.

Numerical Analysis
Characterizing GSVD by singular value expansion of linear operators and its computation

Haibo Li

2024-03-31

Abstract:The generalized singular value decomposition (GSVD) of a matrix pair $\{A, L\}$ with $A\in\mathbb{R}^{m\times n}$ and $L\in\mathbb{R}^{p\times n}$ generalizes the singular value decomposition (SVD) of a single matrix. In this paper, we provide a new understanding of GSVD from the viewpoint of SVD, based on which we propose a new iterative method for computing nontrivial GSVD components of a large-scale matrix pair. By introducing two linear operators $\mathcal{A}$ and $\mathcal{L}$ induced by $\{A, L\}$ between two finite-dimensional Hilbert spaces and applying the theory of singular value expansion (SVE) for linear compact operators, we show that the GSVD of $\{A, L\}$ is nothing but the SVEs of $\mathcal{A}$ and $\mathcal{L}$. This result characterizes completely the structure of GSVD for any matrix pair with the same number of columns. As a direct application of this result, we generalize the standard Golub-Kahan bidiagonalization (GKB) that is a basic routine for large-scale SVD computation such that the resulting generalized GKB (gGKB) process can be used to approximate nontrivial extreme GSVD components of $\{A, L\}$, which is named the gGKB\_GSVD algorithm. We use the GSVD of $\{A, L\}$ to study several basic properties of gGKB and also provide preliminary results about convergence and accuracy of gGKB\_GSVD for GSVD computation. Numerical experiments are presented to demonstrate the effectiveness of this method.

Numerical Analysis
Towards a more robust algorithm for computing the restricted singular value decomposition

Ian N. Zwaan

DOI: https://doi.org/10.48550/arXiv.2002.04828

2020-02-12

Abstract:A new algorithm to compute the restricted singular value decomposition of dense matrices is presented. Like Zha's method \cite{Zha92}, the new algorithm uses an implicit Kogbetliantz iteration, but with four major innovations. The first innovation is a useful quasi-upper triangular generalized Schur form that just requires orthonormal transformations to compute. Depending on the application, this Schur form can be used instead of the full decomposition. The second innovation is a new preprocessing phase that requires fewer rank determinations than previous methods. The third innovation is a numerically stable RSVD algorithm for $2\times 2$ upper-triangular matrices, which forms a key component of the implicit Kogbetliantz iteration. The fourth innovation is an alternative scaling for the restricted singular triplets that results in elegant formulas for their computation. Beyond these four innovations, the qualitative (numerical) characteristics of the algorithm are discussed extensively. Some numerical challenges in the (optional) postprocessing phase are considered too; though, their solutions require further research. Numerical tests and examples confirm the effectiveness of the method.

Numerical Analysis
On Choices of Formulations of Computing the Generalized Singular Value Decomposition of a Large Matrix Pair.

Huang Jinzhi,Jia Zhongxiao

DOI: https://doi.org/10.1007/s11075-020-00984-9

IF: 2.37

2020-01-01

Numerical Algorithms

Abstract:For the computation of the generalized singular value decomposition (GSVD) of a large matrix pair (A,B) of full column rank, the GSVD is commonly formulated as two mathematically equivalent generalized eigenvalue problems, so that a generalized eigensolver can be applied to one of them and the desired GSVD components are then recovered from the computed generalized eigenpairs. Our concern in this paper is, in finite precision arithmetic, which generalized eigenvalue formulation is numerically preferable to compute the desired GSVD components more accurately. We make a detailed perturbation analysis on the two formulations and show how to make a suitable choice between them. Numerical experiments illustrate the results obtained.
On Properties and Structure of the Analytic Singular Value Decomposition

Stephan Weiss,Ian K. Proudler,Giovanni Barbarino,Jennifer Pestana,John G. McWhirter

DOI: https://doi.org/10.1109/tsp.2024.3387726

IF: 4.875

2024-05-10

IEEE Transactions on Signal Processing

Abstract:We investigate the singular value decomposition (SVD) of a rectangular matrix of functions that are analytic on an annulus that includes at least the unit circle. Such matrices occur, e.g., as matrices of transfer functions representing broadband multiple-input multiple-output systems. Our analysis is based on findings for the analytic SVD applicable to continuous time systems, and on the analytic eigenvalue decomposition. Using these, we establish two potentially overlapping cases where analyticity of the SVD factors is denied. Firstly, from a structural point of view, multiplexed systems require oversampling by the multiplexing factor in order to admit an analytic solution. Secondly, from an algebraic perspective, we state under which condition spectral zeros of any singular value require additional oversampling by a factor of two if an analytic solution is to be found. In all other cases, an analytic matrix admits an analytic SVD, whereby the singular values are unique up to a permutation, and the left- and right-singular vectors are coupled through a joint ambiguity w.r.t. an arbitrary allpass function. We demonstrate how some state-of-the-art polynomial matrix decomposition algorithms approximate this solution, motivating the need for dedicated algorithms.

engineering, electrical & electronic
Making matrices better: Geometry and topology of polar and singular value decomposition

Dennis DeTurck,Amora Elsaify,Herman Gluck,Benjamin Grossmann,Joseph Hoisington,Anusha M.Krishnan,Jianru Zhang

DOI: https://doi.org/10.48550/arXiv.1702.02131

2017-02-08

Abstract:Our goal here is to see the space of matrices of a given size from a geometric and topological perspective, with emphasis on the families of various ranks and how they fit together. We pay special attention to the nearest orthogonal neighbor and nearest singular neighbor of a given matrix, both of which play central roles in matrix decompositions, and then against this visual backdrop examine the polar and singular value decompositions and some of their applications.

Rings and Algebras,Numerical Analysis
A Power Method for Computing Singular Value Decomposition

Doulaye Dembele

2024-10-31

Abstract:The singular value decomposition (SVD) allows to put a matrix as a product of three matrices: a matrix with the left singular vectors, a matrix with the positive-valued singular values and a matrix with the right singular vectors. There are two main approaches allowing to get the SVD result: the classical method and the randomized method. The analysis of the classical approach leads to accurate singular values. The randomized approach is especially used for high dimensional matrix and is based on the approximation accuracy without computing necessary all singular values. In this paper, the SVD computation is formalized as an optimization problem and a use of the gradient search algorithm. That results in a power method allowing to get all or the first largest singular values and their associated right vectors. In this iterative search, the accuracy on the singular values and the associated vector matrix depends on the user settings. Two applications of the SVD are the principal component analysis and the autoencoder used in the neural network models.

Numerical Analysis
New discovery of a conversion of generalized singular values

Weiwei Xu,Yingzhou Lee

DOI: https://doi.org/10.48550/arXiv.2106.04367

2021-06-07

Abstract:In this paper, we give a new discovery of conversion of generalized singular values. New formulation is proposed.

General Mathematics
Fifty Three Matrix Factorizations: A systematic approach

Alan Edelman,Sungwoo Jeong

DOI: https://doi.org/10.48550/arXiv.2104.08669

2022-02-15

Abstract:The success of matrix factorizations such as the singular value decomposition (SVD) has motivated the search for even more factorizations. We catalog 53 matrix factorizations, most of which we believe to be new. Our systematic approach, inspired by the generalized Cartan decomposition of Lie theory, also encompasses known factorizations such as the SVD, the symmetric eigendecomposition, the CS decomposition, the hyperbolic SVD, structured SVDs, the Takagi factorization, and others thereby covering familiar matrix factorizations as well as ones that were waiting to be discovered. We suggest that Lie theory has one way or another been lurking hidden in the foundations of the very successful field of matrix computations with applications routinely used in so many areas of computation. In this paper, we investigate consequences of the Cartan decomposition and the little known generalized Cartan decomposition for matrix factorizations. We believe that these factorizations once properly identified can lead to further work on algorithmic computations and applications.

Numerical Analysis
A Hybrid Algorithm for Computing a Partial Singular Value Decomposition Satisfying a Given Threshold

James Baglama,Jonathan A. Chávez Casillas,Vasilije Perović

2024-08-03

Abstract:In this paper, we describe a new hybrid algorithm for computing all singular triplets above a given threshold and provide its implementation in MATLAB/Octave and R. The high performance of our codes and ease at which they can be used, either independently or within a larger numerical scheme, are illustrated through several numerical examples with applications to matrix completion and image compression. Well-documented MATLAB and R codes are provided for public use.

Numerical Analysis
Dominant Singular Value Decomposition Representation for Face Recognition.

Jiwen Lu,Yongwei Zhao

DOI: https://doi.org/10.1016/j.sigpro.2009.11.028

IF: 4.729

2009-01-01

Signal Processing

Abstract:We propose in this paper a new dominant singular value decomposition representation (DSVDR) method for face recognition. Motivated by the fact that each grayscale face image can be decomposed into a composition of a set of bases by the well-known singular value decomposition (SVD) technique and each basis contains different discriminative and reconstructive information for face representation, we present a new face representation method to select a subset of important bases and regulate their singular values (SVs) according to their discriminative and reconstructive power simultaneously for face recognition. Experimental results demonstrate the efficacy of the proposed method.
The Generalized Triangular Decomposition.

Yi Jiang,William W. Hager,Jian Li

DOI: https://doi.org/10.1090/s0025-5718-07-02014-5

IF: 2.118

2007-01-01

Mathematics of Computation

Abstract:Given a complex matrix H, we consider the decomposition H = QRP*, where R is upper triangular and Q and P have orthonormal columns. Special instances of this decomposition include the singular value decomposition (SVD) and the Schur decomposition where R is an upper triangular matrix with the eigenvalues of H on the diagonal. We show that any diagonal for R can be achieved that satisfies Weyl's multiplicative majorization conditions:(k)Pi(i =1) |r(i)| <= (k)Pi(i =1) sigma(i), 1 <= k < K, (k)Pi(i =1) |r(i)| = (k)Pi(i =1) sigma(i),where K is the rank of H, sigma(i) is the i-th largest singular value of H, and r(i) is the i-th largest (in magnitude) diagonal element of R. Given a vector r which satisfies Weyl's conditions, we call the decomposition H = QRP*, where R is upper triangular with prescribed diagonal r, the generalized triangular decomposition (GTD). A direct (nonrecursive) algorithm is developed for computing the GTD. This algorithm starts with the SVD and applies a series of permutations and Givens rotations to obtain the GTD. The numerical stability of the GTD update step is established. The GTD can be used to optimize the power utilization of a communication channel, while taking into account quality of service requirements for subchannels. Another application of the GTD is to inverse eigenvalue problems where the goal is to construct matrices with prescribed eigenvalues and singular values.
A general theory of singular values with applications to signal denoising

Harm Derksen

DOI: https://doi.org/10.48550/arXiv.1705.10881

2017-05-31

Abstract:We study the Pareto frontier for two competing norms $\|\cdot\|_X$ and $\|\cdot\|_Y$ on a vector space. For a given vector $c$, the pareto frontier describes the possible values of $(\|a\|_X,\|b\|_Y)$ for a decomposition $c=a+b$. The singular value decomposition of a matrix is closely related to the Pareto frontier for the spectral and nuclear norm. We will develop a general theory that extends the notion of singular values of a matrix to arbitrary finite dimensional euclidean vector spaces equipped with dual norms. This also generalizes the diagonal singular value decompositions for tensors introduced by the author in previous work. We can apply the results to denoising, where $c$ is a noisy signal, $a$ is a sparse signal and $b$ is noise. Applications include 1D total variation denoising, 2D total variation Rudin-Osher-Fatemi image denoising, LASSO, basis pursuit denoising and tensor decompositions.

Numerical Analysis,Information Theory,Optimization and Control,Statistics Theory
Sparse Factor Analysis for Categorical Data with the Group-Sparse Generalized Singular Value Decomposition

Ju-Chi Yu,Julie Le Borgne,Anjali Krishnan,Arnaud Gloaguen,Cheng-Ta Yang,Laura A Rabin,Hervé Abdi,Vincent Guillemot

2024-09-18

Abstract:Correspondence analysis, multiple correspondence analysis and their discriminant counterparts (i.e., discriminant simple correspondence analysis and discriminant multiple correspondence analysis) are methods of choice for analyzing multivariate categorical data. In these methods, variables are integrated into optimal components computed as linear combinations whose weights are obtained from a generalized singular value decomposition (GSVD) that integrates specific metric constraints on the rows and columns of the original data matrix. The weights of the linear combinations are, in turn, used to interpret the components, and this interpretation is facilitated when components are 1) pairwise orthogonal and 2) when the values of the weights are either large or small but not intermediate-a pattern called a simple or a sparse structure. To obtain such simple configurations, the optimization problem solved by the GSVD is extended to include new constraints that implement component orthogonality and sparse weights. Because multiple correspondence analysis represents qualitative variables by a set of binary variables, an additional group constraint is added to the optimization problem in order to sparsify the whole set representing one qualitative variable. This new algorithm-called group-sparse GSVD (gsGSVD)-integrates these constraints via an iterative projection scheme onto the intersection of subspaces where each subspace implements a specific constraint. In this paper, we expose this new algorithm and show how it can be adapted to the sparsification of simple and multiple correspondence analysis, and illustrate its applications with the analysis of four different data sets-each illustrating the sparsification of a particular CA-based analysis.

Statistics Theory

Generalized Singular Value Decompositions: A proposal for a standardized nomenclature

A Conditional Singular Value Decomposition

A Higher-Order Generalized Singular Value Decomposition for Rank Deficient Matrices

Singular value decomposition of complexes

High Order Numerical Methods To Approximate The Singular Value Decomposition

Randomized dual singular value decomposition

The Singular Value Decomposition, Applications and Beyond.

Singular Value Decomposition Approximation via Kronecker Summations for Imaging Applications

Characterizing GSVD by singular value expansion of linear operators and its computation

Towards a more robust algorithm for computing the restricted singular value decomposition

On Choices of Formulations of Computing the Generalized Singular Value Decomposition of a Large Matrix Pair.

On Properties and Structure of the Analytic Singular Value Decomposition

Making matrices better: Geometry and topology of polar and singular value decomposition

A Power Method for Computing Singular Value Decomposition

New discovery of a conversion of generalized singular values

Fifty Three Matrix Factorizations: A systematic approach

A Hybrid Algorithm for Computing a Partial Singular Value Decomposition Satisfying a Given Threshold

Dominant Singular Value Decomposition Representation for Face Recognition.

The Generalized Triangular Decomposition.

A general theory of singular values with applications to signal denoising

Sparse Factor Analysis for Categorical Data with the Group-Sparse Generalized Singular Value Decomposition