On the Distribution of GSVD

Zhuo Chen,Zhiguo Ding

DOI: https://doi.org/10.48550/arXiv.1902.09134

2019-02-25

Abstract:In this paper, some new results on the distribution of the generalized singular value decomposition (GSVD) are presented.

Information Theory

What problem does this paper attempt to address?

More General Results on Mixed Extreme Value Distributions

Jiang Yue-xiang

DOI: https://doi.org/10.1631/jzus.2005.a0769

2005-01-01

Journal of Zhejiang University SCIENCE A

Abstract:The sequences { Z i,n , 1≤ i ≤ n }, n ≥1 are multi-nornial distribution among i.i.d. random variables { X , i , i≥1} , { X 2, i, , i≥1} ,…, { X m,i ≥1} . The extreme value distribution G x (x) of this particular triangular array of i.i.d. random variables Z 1, n , Z 2, n ,…, Z n,n is discussed. A new type of not max-stable extreme value distributions which are Fréchet mixture, Gumbel mixture and Weibull mixture has been found if F j , …,F m belong to the same MDA. Whether mixtures of different types of extreme value distributions exist or not and the more general case are discussed in this paper. We found that G Z ( x ) does not exist as mixture forms of the different types of extreme value distributions, after we investigated all cases.
Two-sided uniformly randomized GSVD for large-scale discrete ill-posed problems with Tikhonov regularizations

Weiwei Xu,Weijie Shen,Zheng-Jian Bai

2024-12-10

Abstract:The generalized singular value decomposition (GSVD) is a powerful tool for solving discrete ill-posed problems. In this paper, we propose a two-sided uniformly randomized GSVD algorithm for solving the large-scale discrete ill-posed problem with the general Tikhonov regularization. Based on two-sided uniform random sampling, the proposed algorithm can improve the efficiency with less computing time and memory requirement and obtain expected accuracy. The error analysis for the proposed algorithm is also derived. Finally, we report some numerical examples to illustrate the efficiency of the proposed algorithm.

Numerical Analysis
Tikhonov Regularization and Randomized GSVD.

Yimin Wei,Pengpeng Xie,Liping Zhang

DOI: https://doi.org/10.1137/15M1030200

IF: 1.908

2016-01-01

SIAM Journal on Matrix Analysis and Applications

Abstract:The generalized singular value decomposition (GSVD) is one of the essential tools in numerical linear algebra. This paper proposes a regularization method, combining Tikhonov regularization in general form with the truncated GSVD. Then the randomized algorithms are adopted to implement the truncation process. This randomized GSVD for the regularization of the large-scale ill-posed problems can achieve good accuracy with less computational time and memory requirement than the classical regularization methods. Finally, we present the error analyses for the randomized algorithms. Some illustrative numerical examples are provided.
Modifying the Generalized Singular Value Decomposition with Application with Application in Direction-of-arrival Finding

Hongyuan Zha,Zhenyue Zhang

DOI: https://doi.org/10.1007/bf02510925

1998-01-01

Abstract:We consider updating and downdating problems for the generalized singular value decomposition (GSVD) of matrix pairs when new rows are added to one of the matrices or old rows are deleted. Two classes of algorithms are developed, one based on the CS decomposition formulation of the GSVD and the other based on the generalized eigenvalue decomposition formulation. In both cases we show that the updating and downdating problems can be reduced to a rank-one SVD updating problem. We also provide perturbation analysis for the cases when the added or deleted rows are subject to errors. Numerical experiments on direction-of-arrival (DOA) finding with colored noises are carried out to demonstrate the tracking ability of the algorithms.
Characterizing GSVD by singular value expansion of linear operators and its computation

Haibo Li

2024-03-31

Abstract:The generalized singular value decomposition (GSVD) of a matrix pair $\{A, L\}$ with $A\in\mathbb{R}^{m\times n}$ and $L\in\mathbb{R}^{p\times n}$ generalizes the singular value decomposition (SVD) of a single matrix. In this paper, we provide a new understanding of GSVD from the viewpoint of SVD, based on which we propose a new iterative method for computing nontrivial GSVD components of a large-scale matrix pair. By introducing two linear operators $\mathcal{A}$ and $\mathcal{L}$ induced by $\{A, L\}$ between two finite-dimensional Hilbert spaces and applying the theory of singular value expansion (SVE) for linear compact operators, we show that the GSVD of $\{A, L\}$ is nothing but the SVEs of $\mathcal{A}$ and $\mathcal{L}$. This result characterizes completely the structure of GSVD for any matrix pair with the same number of columns. As a direct application of this result, we generalize the standard Golub-Kahan bidiagonalization (GKB) that is a basic routine for large-scale SVD computation such that the resulting generalized GKB (gGKB) process can be used to approximate nontrivial extreme GSVD components of $\{A, L\}$, which is named the gGKB\_GSVD algorithm. We use the GSVD of $\{A, L\}$ to study several basic properties of gGKB and also provide preliminary results about convergence and accuracy of gGKB\_GSVD for GSVD computation. Numerical experiments are presented to demonstrate the effectiveness of this method.

Numerical Analysis
On Choices of Formulations of Computing the Generalized Singular Value Decomposition of a Large Matrix Pair.

Huang Jinzhi,Jia Zhongxiao

DOI: https://doi.org/10.1007/s11075-020-00984-9

IF: 2.37

2020-01-01

Numerical Algorithms

Abstract:For the computation of the generalized singular value decomposition (GSVD) of a large matrix pair (A,B) of full column rank, the GSVD is commonly formulated as two mathematically equivalent generalized eigenvalue problems, so that a generalized eigensolver can be applied to one of them and the desired GSVD components are then recovered from the computed generalized eigenpairs. Our concern in this paper is, in finite precision arithmetic, which generalized eigenvalue formulation is numerically preferable to compute the desired GSVD components more accurately. We make a detailed perturbation analysis on the two formulations and show how to make a suitable choice between them. Numerical experiments illustrate the results obtained.
The Application of Spectral Distribution of Product Matrices of Large Dimensional Random Matrices in the Factor Analysis

B. Jin,B. Miao,Wuyi Ye,Zhen-xiang Wu

2006-01-01

Abstract:Abstract In the factor analysis model with large cross-section and time-series dimensions ,we propose a new method to estimate the number of factors . Specially if the idiosyncratic terms satisfy a linear time series model, the estimators of the parameters in the time series model can be made . The theoretical properties of the estimators are also explored.A simulation study and an empirical analysis are conducted
The Joint Bidiagonalization Method for Large GSVD Computations in Finite Precision.

Zhongxiao Jia,Haibo Li

DOI: https://doi.org/10.1137/22m1483608

IF: 1.908

2023-01-01

SIAM Journal on Matrix Analysis and Applications

Abstract:The joint bidiagonalization (JBD) method has been used to compute some extreme generalized singular values and vectors of a large regular matrix pair {A,L}, where we propose three approaches to compute approximate generalized singular values and vectors. We make a numerical analysis of the underlying JBD process and establish relationships between it and two mathematically equivalent Lanczos bidiagonalizations in finite precision. Based on the results of numerical analysis, we investigate the convergence of the approximate generalized singular values and vectors of {A,L}. The results show that, under some mild conditions, the semiorthogonality of Lanczos type vectors suffices to deliver approximate generalized singular values with the same accuracy as the full orthogonality does, meaning that it is only necessary to seek for efficient semiorthogonalization strategies for the JBD process. We also establish a sharp bound for the residual norm of an approximate generalized singular value and corresponding approximate right generalized singular vectors, which can reliably estimate the residual norm without explicitly computing the approximate right generalized singular vectors before the convergence occurs.
Generalized Semi-Invariant Distributions on P-Adic S Paces

Jiuzu Hong,Binyong Sun

DOI: https://doi.org/10.1007/s00208-016-1444-8

2017-01-01

Mathematische Annalen

Abstract:In this paper we investigate some methods on calculating the spaces of generalized semi-invariant distributions on p-adic spaces. Using homological methods, we give a criterion of automatic extension of (generalized) semi-invariant distributions. Based on the meromorphic continuations of Igusa zeta integrals, we give another criteria with purely algebraic geometric conditions, on the extension of generalized semi-invariant distributions.
Perturbation Analysis of Randomized SVD and its Applications to High-dimensional Statistics

Yichi Zhang,Minh Tang

DOI: https://doi.org/10.48550/arXiv.2203.10262

2022-08-02

Abstract:Randomized singular value decomposition (RSVD) is a class of computationally efficient algorithms for computing the truncated SVD of large data matrices. Given a $n \times n$ symmetric matrix $\mathbf{M}$, the prototypical RSVD algorithm outputs an approximation of the $k$ leading singular vectors of $\mathbf{M}$ by computing the SVD of $\mathbf{M}^{g} \mathbf{G}$; here $g \geq 1$ is an integer and $\mathbf{G} \in \mathbb{R}^{n \times k}$ is a random Gaussian sketching matrix. In this paper we study the statistical properties of RSVD under a general "signal-plus-noise" framework, i.e., the observed matrix $\hat{\mathbf{M}}$ is assumed to be an additive perturbation of some true but unknown signal matrix $\mathbf{M}$. We first derive upper bounds for the $\ell_2$ (spectral norm) and $\ell_{2\to\infty}$ (maximum row-wise $\ell_2$ norm) distances between the approximate singular vectors of $\hat{\mathbf{M}}$ and the true singular vectors of the signal matrix $\mathbf{M}$. These upper bounds depend on the signal-to-noise ratio (SNR) and the number of power iterations $g$. A phase transition phenomenon is observed in which a smaller SNR requires larger values of $g$ to guarantee convergence of the $\ell_2$ and $\ell_{2\to\infty}$ distances. We also show that the thresholds for $g$ where these phase transitions occur are sharp whenever the noise matrices satisfy a certain trace growth condition. Finally, we derive normal approximations for the row-wise fluctuations of the approximate singular vectors and the entrywise fluctuations of the approximate matrix. We illustrate our theoretical results by deriving nearly-optimal performance guarantees for RSVD when applied to three statistical inference problems, namely, community detection, matrix completion, and principal component analysis with missing data.

Statistics Theory,Numerical Analysis,Computation,Machine Learning
The Singular Value Decomposition, Applications and Beyond.

Zhihua Zhang

DOI: https://doi.org/10.48550/arxiv.1510.08532

2015-01-01

Abstract:The singular value decomposition (SVD) is not only a classical theory in matrix computation and analysis, but also is a powerful tool in machine learning and modern data analysis. In this tutorial we first study the basic notion of SVD and then show the central role of SVD in matrices. Using majorization theory, we consider variational principles of singular values and eigenvalues. Built on SVD and a theory of symmetric gauge functions, we discuss unitarily invariant norms, which are then used to formulate general results for matrix low rank approximation. We study the subdifferentials of unitarily invariant norms. These results would be potentially useful in many machine learning problems such as matrix completion and matrix data classification. Finally, we discuss matrix low rank approximation and its recent developments such as randomized SVD, approximate matrix multiplication, CUR decomposition, and Nystrom approximation. Randomized algorithms are important approaches to large scale SVD as well as fast matrix computations.
On the Power of SVD in the Stochastic Block Model

Xinyu Mao,Jiapeng Zhang

DOI: https://doi.org/10.48550/arXiv.2309.15322

2023-09-27

Abstract:A popular heuristic method for improving clustering results is to apply dimensionality reduction before running clustering algorithms. It has been observed that spectral-based dimensionality reduction tools, such as PCA or SVD, improve the performance of clustering algorithms in many applications. This phenomenon indicates that spectral method not only serves as a dimensionality reduction tool, but also contributes to the clustering procedure in some sense. It is an interesting question to understand the behavior of spectral steps in clustering problems. As an initial step in this direction, this paper studies the power of vanilla-SVD algorithm in the stochastic block model (SBM). We show that, in the symmetric setting, vanilla-SVD algorithm recovers all clusters correctly. This result answers an open question posed by Van Vu (Combinatorics Probability and Computing, 2018) in the symmetric setting.

Machine Learning,Data Structures and Algorithms,Probability
Bulk and Soft-Edge Universality for Singular Values of Products of Ginibre Random Matrices

Dang-Zheng Liu,Dong Wang,Lun Zhang

DOI: https://doi.org/10.1214/15-aihp696

2016-01-01

Annales de l Institut Henri Poincaré Probabilités et Statistiques

Abstract:It has been shown by Akemann, Ipsen and Kieburg that the squared singular values of products of $M$ rectangular random matrices with independent complex Gaussian entries are distributed according to a determinantal point process with a correlation kernel that admits a representation in terms of Meijer G-functions. We prove the universality of the local statistics of the squared singular values, namely, the bulk universality given by the sine kernel and the edge universality given by the Airy kernel. The proof is based on the asymptotic analysis for the double contour integral representation of the correlation kernel. Our strategy can be generalized to deal with other models of products of random matrices introduced recently and to establish similar universal results. Two more examples are investigated, one is the product of $M$ Ginibre matrices and the inverse of $K$ Ginibre matrices studied by Forrester, and the other one is the product of $M-1$ Ginibre matrices with one truncated unitary matrix considered by Kuijlaars and Stivigny.
On the Structures of the Nonsingular Matrices of QQ-SVD

GUO Wen-bin,ZHOU Shang-qi,ZHANG Li-men

DOI: https://doi.org/10.3969/j.issn.1672-6634.2005.01.005

2005-01-01

Abstract:Matrix dcomposition plays an important role in the study of matrix theory, especially singular value decomposition (SVD). In this article we study the structures of the nonsingular matrices of the multiple quotient singular value decomposition QQ-SVD.
Generalized Singular Value Thresholding

Canyi Lu,Changbo Zhu,Chunyan Xu,Shuicheng Yan,Zhouchen Lin

DOI: https://doi.org/10.1609/aaai.v29i1.9464

2015-01-01

Abstract:This work studies the Generalized Singular Value Thresholding (GSVT) operator Prox(g)(sigma) (.),Prox(g)(sigma) (B) = arg minX Sigma(m)(i =1) g(sigma(i) (X)) vertical bar 1/2 vertical bar vertical bar X -B vertical bar vertical bar(2)(F),associated with a nonconvex function g defined on the singular values of X. We prove that GSVT can be obtained by performing the proximal operator of g (denoted as Prox(g) (.)) on the singular values since Prox(g) (.) is monotone when g is lower bounded. If the nonconvex g satisfies some conditions (many popular nonconvex surrogate functions, e.g., l(p)-norm, 0 < p < 1, of l(0)-norm are special cases), a general solver to find Prox(g) (b) is proposed for any b >= 0. GSVT greatly generalizes the known Singular Value Thresholding (SVT) which is a basic subroutine in many convex low rank minimization methods. We are able to solve the nonconvex low rank minimization problem by using GSVT in place of SVT.
On Generalized Sub-Gaussian Canonical Processes and Their Applications

Yiming Chen,Yuxuan Wang,Kefan Zhu

2024-02-01

Abstract:We obtain the tail probability of generalized sub-Gaussian canonical processes. It can be viewed as a variant of the Bernstein-type inequality in the i.i.d case, and we further get a tighter bound of concentration inequality through uniformly randomized techniques. A concentration inequality for general functions involving independent random variables is also derived as an extension. As for applications, we derive convergence results for principal component analysis and the Rademacher complexities method.

Probability
Miscellaneous results related to the Gaussian product inequality conjecture for the joint distribution of traces of Wishart matrices

Christian Genest,Frédéric Ouimet

DOI: https://doi.org/10.1016/j.jmaa.2022.126951

2023-01-25

Abstract:This note reports partial results related to the Gaussian product inequality (GPI) conjecture for the joint distribution of traces of Wishart matrices. In particular, several GPI-related results from Wei (2014) and Liu et al. (2015) are extended in two ways: by replacing the power functions with more general classes of functions, and by replacing the usual Gaussian and multivariate gamma distributional assumptions by the more general trace-Wishart distribution assumption. These findings suggest that a Kronecker product form of the GPI holds for diagonal blocks of any Wishart distribution.

Probability,Functional Analysis,Statistics Theory
The distribution of the product of independent variance-gamma random variables

Robert E. Gaunt,Siqi Li

2024-05-11

Abstract:Let $X$ and $Y$ be independent variance-gamma random variables with zero location parameter; then the exact probability density function of the product $XY$ is derived. Some basic distributional properties are also derived, including formulas for the cumulative distribution function and the characteristic function, as well as asymptotic approximations for the density, tail probabilities and the quantile function. As special cases, we deduce some key distributional properties for the product of two independent asymmetric Laplace random variables as well as the product of four jointly correlated zero mean normal random variables with a particular block diagonal covariance matrix. As a by-product of our analysis, we deduce some new reduction formulas for the Meijer $G$-function.

Probability
Comparison Theorems for the Distribution of Quadratic Forms of Gaussian Vectors and Cauchy Vectors

PAN Jin-yu,ZHANG Xin-sheng

DOI: https://doi.org/10.3969/j.issn.0427-7104.2006.05.003

2006-01-01

Abstract:Based on the known result of Bakirov,further discussions on the comparison for the distribution of quadratic form of Gaussian vectors and Cauchy vectors are made.Some new inequalities on the distribution of quadratic forms of Gaussian vectors and Cauchy vectors are obtained.
Distributions in spaces with thick submanifolds

Jiajia Ding,Jasson Vindas,Yunyun Yang

2024-08-06

Abstract:This article generalizes the results of [J. Math. Anal. Appl. 512 (2022), Article No. 126075], which presented a theory of distributions (generalized functions) with a singular curve contained in the domain of the test functions. In this present article we construct a theory of distributions in $\mathbb{R}^n$ with a ``thick submanifold'', that is, a new theory of thick distributions in $\mathbb{R}^n$ whose domain contains a submanifold on which test functions may be singular.

Functional Analysis,Analysis of PDEs,Differential Geometry

On the Distribution of GSVD

More General Results on Mixed Extreme Value Distributions

Two-sided uniformly randomized GSVD for large-scale discrete ill-posed problems with Tikhonov regularizations

Tikhonov Regularization and Randomized GSVD.

Modifying the Generalized Singular Value Decomposition with Application with Application in Direction-of-arrival Finding

Characterizing GSVD by singular value expansion of linear operators and its computation

On Choices of Formulations of Computing the Generalized Singular Value Decomposition of a Large Matrix Pair.

The Application of Spectral Distribution of Product Matrices of Large Dimensional Random Matrices in the Factor Analysis

The Joint Bidiagonalization Method for Large GSVD Computations in Finite Precision.

Generalized Semi-Invariant Distributions on P-Adic S Paces

Perturbation Analysis of Randomized SVD and its Applications to High-dimensional Statistics

The Singular Value Decomposition, Applications and Beyond.

On the Power of SVD in the Stochastic Block Model

Bulk and Soft-Edge Universality for Singular Values of Products of Ginibre Random Matrices

On the Structures of the Nonsingular Matrices of QQ-SVD

Generalized Singular Value Thresholding

On Generalized Sub-Gaussian Canonical Processes and Their Applications

Miscellaneous results related to the Gaussian product inequality conjecture for the joint distribution of traces of Wishart matrices

The distribution of the product of independent variance-gamma random variables

Comparison Theorems for the Distribution of Quadratic Forms of Gaussian Vectors and Cauchy Vectors

Distributions in spaces with thick submanifolds