Abstract:In many applications one needs to process massive data of high dimensionality. In order to make data compact and reduce computational complexity, dimensionality reduction algorithms are commonly used. Linear discriminant analysis (LDA) is one of the most used approaches, which leads to a matrix trace ratio problem, i.e., maximization of the trace ratio rho(V)= Tr(V(T)AV)/Tr((VBV)-B-T), where A and B are nxn real symmetric matrices with B positive definite, and V is an nxp column orthonormal matrix. In this paper, we consider a commonly used Iterative Trace Ratio (ITR) algorithm developed by Ngo et al., The trace ratio optimization problem, SIAM Review, 54 (3) (2012), pp. 545-569. In implementations, it is common to use the symmetric Lanczos method to compute the p eigenvectors of a certain large matrix corresponding to its p largest eigenvalues at each iteration, and the resulting algorithm is abbreviated as ITR L. We establish the global convergence and local quadratic convergence of the trace ratio itself. We then make two improvements over ITR L: (i) using the refined Lanczos method to compute the desired eigenvectors at each iteration and (ii) providing a better initial guess via solving a generalized eigenvalue problem of the matrix pair (A, B). The resulting algorithms are abbreviated as ITR RL and ITR GeigRL, respectively. Numerical experiments demonstrate that ITR RL and ITR GeigRL outperform ITR L substantially.

Trace Inequalities for Matrix Products and Trace Bounds for the Solution of the Algebraic Riccati Equations

Extensions of some matrix inequalities related to trace and partial traces

Partial trace inequalities for partial transpose of positive semidefinite block matrices

Inequalities regarding partial trace and partial determinant

Comments on "upper Matrix Bounds for the Discrete Algebraic Riccati Matrix Equation" [with Author's Reply]

Inequalities of singular values and unitarily invariant norms for sums and products of matrices

Refined von Neumann-type trace inequality and its application

A subspace method for large-scale trace ratio problems

A Convergence Analysis on the Iterative Trace Ratio Algorithm and Its Refinements

On matrix analogs of Fermat’s little theorem

Trace Inequalities, Isocapacitary Inequalities, and Regularity of the Complex Hessian Equations

Counting $2\times2$ integer matrices with fixed trace and determinant

Double-variable trace maximization for extreme generalized singular quartets of a matrix pair: A geometric method

Computation of the von Neumann entropy of large matrices via trace estimators and rational Krylov methods

On the trace approximations of products of Toeplitz matrices

Closing the duality gap of the generalized trace ratio problem

Miscellaneous results related to the Gaussian product inequality conjecture for the joint distribution of traces of Wishart matrices

Trace Inequalities and Kinematic Metrics

A note on algebraic Riccati equations associated with reducible singular M-matrices

A Rank Equality Of Matrices And Its Application

Matrix algebras with degenerate traces and trace identities