Abstract:It is increasingly interesting to model the relationship between two sets of high‐dimensional measurements with potentially high correlations. Canonical correlation analysis (CCA) is a classical tool that explores the dependency of two multivariate random variables and extracts canonical pairs of highly correlated linear combinations. Driven by applications in genomics, text mining, and imaging research, among others, many recent studies generalize CCA to high‐dimensional settings. However, most of them either rely on strong assumptions on covariance matrices, or do not produce nested solutions. We propose a new sparse CCA (SCCA) method that recasts high‐dimensional CCA as an iterative penalized least squares problem. Thanks to the new iterative penalized least squares formulation, our method directly estimates the sparse CCA directions with efficient algorithms. Therefore, in contrast to some existing methods, the new SCCA does not impose any sparsity assumptions on the covariance matrices. The proposed SCCA is also very flexible in the sense that it can be easily combined with properly chosen penalty functions to perform structured variable selection and incorporate prior information. Moreover, our proposal of SCCA produces nested solutions and thus provides great convenient in practice. Theoretical results show that SCCA can consistently estimate the true canonical pairs with an overwhelming probability in ultra‐high dimensions. Numerical results also demonstrate the competitive performance of SCCA.

Tssnalm: A Fast Two-Stage Semi-Smooth Newton Augmented Lagrangian Method for Sparse CCA.

On Sparse Canonical Correlation Analysis

Sparse Generalized Canonical Correlation Analysis: Distributed Alternating Iteration Based Approach

An iterative penalized least squares approach to sparse canonical correlation analysis

An Inexact Manifold Augmented Lagrangian Method for Adaptive Sparse Canonical Correlation Analysis with Trace Lasso Regularization

An Efficient Newton-Based Method for Sparse Generalized Canonical Correlation Analysis

Exploring Shared Subspace and Joint Sparsity for Canonical Correlation Analysis.

Sparse Canonical Correlation Analysis: New Formulation and Algorithm.

An Alternating Manifold Proximal Gradient Method for Sparse PCA and Sparse CCA

Sparse Weighted Canonical Correlation Analysis

Imbalanced Sparse Canonical Correlation Analysis

ℓ0-Based Sparse Canonical Correlation Analysis with Application to Cross-Language Document Retrieval

Efficient and Fast Joint Sparse Constrained Canonical Correlation Analysis for Fault Detection.

Eigenvector-based sparse canonical correlation analysis: Fast computation for estimation of multiple canonical vectors

Sparse canonical correlation analysis for recognition

Sparse 2-D Canonical Correlation Analysis Via Low Rank Matrix Approximation for Feature Extraction.

A Lagrange–Newton Algorithm for Tensor Sparse Principal Component Analysis

Sparse Kernel Canonical Correlation Analysis Via $\Ell_1$-Regularization

Sufficient Canonical Correlation Analysis

Instance-specific Canonical Correlation Analysis

2D-LPCCA and 2D-SPCCA: Two New Canonical Correlation Methods for Feature Extraction, Fusion and Recognition