Reliability of carpal angle determinations.

M. García-Elias,K. An,P. Amadio,W. Cooney,R. L. Linscheid

DOI: https://doi.org/10.1016/S0363-5023(89)80053-X

1989-11-01

Abstract:

What problem does this paper attempt to address?

Regularized Canonical Correlation Analysis with Unlabeled Data

Xi-chuan Zhou,Hai-bin Shen

DOI: https://doi.org/10.1631/jzus.a0820221

2009-01-01

Journal of Zhejiang University SCIENCE A

Abstract:In standard canonical correlation analysis (CCA), the data from definite datasets are used to estimate their canonical correlation. In real applications, for example in bilingual text retrieval, it may have a great portion of data that we do not know which set it belongs to. This part of data is called unlabeled data, while the rest from definite datasets is called labeled data. We propose a novel method called regularized canonical correlation analysis (RCCA), which makes use of both labeled and unlabeled samples. Specifically, we learn to approximate canonical correlation as if all data were labeled. Then, we describe a generalization of RCCA for the multi-set situation. Experiments on four real world datasets, Yeast, Cloud, Iris, and Haberman, demonstrate that, by incorporating the unlabeled data points, the accuracy of correlation coefficients can be improved by over 30%.
Sparse Generalized Canonical Correlation Analysis: Distributed Alternating Iteration Based Approach.

Kexin Lv,Jia Cai,Junyi Huo,Chao Shang,Xiaolin Huang,Jie Yang

DOI: https://doi.org/10.1162/neco_a_01673

IF: 3.278

2024-01-01

Neural Computation

Abstract:Sparse canonical correlation analysis (CCA) is a useful statistical tool to detect latent information with sparse structures. However, sparse CCA, where the sparsity could be considered as a Laplace prior on the canonical variates, works only for two data sets, that is, there are only two views or two distinct objects. To overcome this limitation, we propose a sparse generalized canonical correlation analysis (GCCA), which could detect the latent relations of multiview data with sparse structures. Specifically, we convert the GCCA into a linear system of equations and impose ℓ1 minimization penalty to pursue sparsity. This results in a nonconvex problem on the Stiefel manifold. Based on consensus optimization, a distributed alternating iteration approach is developed, and consistency is investigated elaborately under mild conditions. Experiments on several synthetic and real-world data sets demonstrate the effectiveness of the proposed algorithm.
Regularised Canonical Correlation Analysis: graphical lasso, biplots and beyond

Lennie Wells,Kumar Thurimella,Sergio Bacallado

2024-03-05

Abstract:Recent developments in regularized Canonical Correlation Analysis (CCA) promise powerful methods for high-dimensional, multiview data analysis. However, justifying the structural assumptions behind many popular approaches remains a challenge, and features of realistic biological datasets pose practical difficulties that are seldom discussed. We propose a novel CCA estimator rooted in an assumption of conditional independencies and based on the Graphical Lasso. Our method has desirable theoretical guarantees and good empirical performance, demonstrated through extensive simulations and real-world biological datasets. Recognizing the difficulties of model selection in high dimensions and other practical challenges of applying CCA in real-world settings, we introduce a novel framework for evaluating and interpreting regularized CCA models in the context of Exploratory Data Analysis (EDA), which we hope will empower researchers and pave the way for wider adoption.

Methodology,Statistics Theory,Applications
Sparse Canonical Correlation Analysis: New Formulation and Algorithm.

Delin Chu,Li-Zhi Liao,Michael K. Ng,Xiaowei Zhang

DOI: https://doi.org/10.1109/tpami.2013.104

IF: 23.6

2013-01-01

IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract:In this paper, we study canonical correlation analysis (CCA), which is a powerful tool in multivariate data analysis for finding the correlation between two sets of multidimensional variables. The main contributions of the paper are: 1) to reveal the equivalent relationship between a recursive formula and a trace formula for the multiple CCA problem, 2) to obtain the explicit characterization for all solutions of the multiple CCA problem even when the corresponding covariance matrices are singular, 3) to develop a new sparse CCA algorithm, and 4) to establish the equivalent relationship between the uncorrelated linear discriminant analysis and the CCA problem. We test several simulated and real-world datasets in gene classification and cross-language document retrieval to demonstrate the effectiveness of the proposed algorithm. The performance of the proposed method is competitive with the state-of-the-art sparse CCA algorithms.
Unconstrained Stochastic CCA: Unifying Multiview and Self-Supervised Learning

James Chapman,Lennie Wells,Ana Lawry Aguila

2024-05-02

Abstract:The Canonical Correlation Analysis (CCA) family of methods is foundational in multiview learning. Regularised linear CCA methods can be seen to generalise Partial Least Squares (PLS) and be unified with a Generalized Eigenvalue Problem (GEP) framework. However, classical algorithms for these linear methods are computationally infeasible for large-scale data. Extensions to Deep CCA show great promise, but current training procedures are slow and complicated. First we propose a novel unconstrained objective that characterizes the top subspace of GEPs. Our core contribution is a family of fast algorithms for stochastic PLS, stochastic CCA, and Deep CCA, simply obtained by applying stochastic gradient descent (SGD) to the corresponding CCA objectives. Our algorithms show far faster convergence and recover higher correlations than the previous state-of-the-art on all standard CCA and Deep CCA benchmarks. These improvements allow us to perform a first-of-its-kind PLS analysis of an extremely large biomedical dataset from the UK Biobank, with over 33,000 individuals and 500,000 features. Finally, we apply our algorithms to match the performance of `CCA-family' Self-Supervised Learning (SSL) methods on CIFAR-10 and CIFAR-100 with minimal hyper-parameter tuning, and also present theory to clarify the links between these methods and classical CCA, laying the groundwork for future insights.

Machine Learning,Artificial Intelligence
A Survey on Canonical Correlation Analysis

Xinghao Yang,Weifeng Liu,Wei Liu,Dacheng Tao

DOI: https://doi.org/10.1109/tkde.2019.2958342

IF: 9.235

2021-06-01

IEEE Transactions on Knowledge and Data Engineering

Abstract:In recent years, the advances in data collection and statistical analysis promotes canonical correlation analysis (CCA) available for more advanced research. CCA is the main technique for two-set data dimensionality reduction such that the correlation between the pairwise variables in the common subspace is mutually maximized. Over 80-years of developments, a number of CCA models have been proposed according to different machine learning mechanisms. However, the field lacks an insightful review for the state-of-art developments. This survey targets to provide a well-organized overview for CCA and its extensions. Specifically, we first review the CCA theory from the perspective of both model formation and model optimization. The association between two popular solution methods, i.e., eigen value decomposition (EVD) and singular value decomposition (SVD), are discussed. Following that, we present a taxonomy of current progresses and classify them into seven groups: 1) multi-view CCA, 2) probabilistic CCA, 3) deep CCA, 4) kernel CCA, 5) discriminative CCA, 6) sparse CCA and 7) locality preserving CCA. For each group, we demonstrate two or three representative mathematical models, identifying their strengths and limitations. We summarize the representative applications and numerical results of these seven groups in real-world practices, collecting the data sets and open-sources for implementation. In the end, we provide several promising future research directions that can improve the current state of the art.

computer science, information systems, artificial intelligence,engineering, electrical & electronic
Sparse Kernel Canonical Correlation Analysis Via $\Ell_1$-Regularization

Xiaowei Zhang,Delin Chu,Li-Zhi Liao,Michael K. Ng

DOI: https://doi.org/10.48550/arxiv.1701.04207

2017-01-01

Abstract:Canonical correlation analysis (CCA) is a multivariate statistical technique for finding the linear relationship between two sets of variables. The kernel generalization of CCA named kernel CCA has been proposed to find nonlinear relations between datasets. Despite their wide usage, they have one common limitation that is the lack of sparsity in their solution. In this paper, we consider sparse kernel CCA and propose a novel sparse kernel CCA algorithm (SKCCA). Our algorithm is based on a relationship between kernel CCA and least squares. Sparsity of the dual transformations is introduced by penalizing the $\ell_{1}$-norm of dual vectors. Experiments demonstrate that our algorithm not only performs well in computing sparse dual transformations but also can alleviate the over-fitting problem of kernel CCA.
On Sparse Canonical Correlation Analysis

Yongchun Li,Santanu S. Dey,Weijun Xie

2023-12-31

Abstract:The classical Canonical Correlation Analysis (CCA) identifies the correlations between two sets of multivariate variables based on their covariance, which has been widely applied in diverse fields such as computer vision, natural language processing, and speech analysis. Despite its popularity, CCA can encounter challenges in explaining correlations between two variable sets within high-dimensional data contexts. Thus, this paper studies Sparse Canonical Correlation Analysis (SCCA) that enhances the interpretability of CCA. We first show that SCCA generalizes three well-known sparse optimization problems, sparse PCA, sparse SVD, and sparse regression, which are all classified as NP-hard problems. This result motivates us to develop strong formulations and efficient algorithms. Our main contributions include (i) the introduction of a combinatorial formulation that captures the essence of SCCA and allows the development of approximation algorithms; (ii) the derivation of an equivalent mixed-integer semidefinite programming model that facilitates a specialized branch-and-cut algorithm with analytical cuts; and (iii) the establishment of the complexity results for two low-rank special cases of SCCA. The effectiveness of our proposed formulations and algorithms is validated through numerical experiments.

Optimization and Control
Canonical Correlation Analysis as Reduced Rank Regression in High Dimensions

Claire Donnat,Elena Tuzhilina

DOI: https://doi.org/10.48550/arXiv.2405.19539

2024-05-30

Abstract:Canonical Correlation Analysis (CCA) is a widespread technique for discovering linear relationships between two sets of variables $X \in \mathbb{R}^{n \times p}$ and $Y \in \mathbb{R}^{n \times q}$. In high dimensions however, standard estimates of the canonical directions cease to be consistent without assuming further structure. In this setting, a possible solution consists in leveraging the presumed sparsity of the solution: only a subset of the covariates span the canonical directions. While the last decade has seen a proliferation of sparse CCA methods, practical challenges regarding the scalability and adaptability of these methods still persist. To circumvent these issues, this paper suggests an alternative strategy that uses reduced rank regression to estimate the canonical directions when one of the datasets is high-dimensional while the other remains low-dimensional. By casting the problem of estimating the canonical direction as a regression problem, our estimator is able to leverage the rich statistics literature on high-dimensional regression and is easily adaptable to accommodate a wider range of structural priors. Our proposed solution maintains computational efficiency and accuracy, even in the presence of very high-dimensional data. We validate the benefits of our approach through a series of simulated experiments and further illustrate its practicality by applying it to three real-world datasets.

Methodology
Comparison of NIRS systems with a motor execution paradigm

M. Schürholz,A. Ehlis,A. Fallgatter,T. Penney,N. Birbaumer,R. Sitaram

Abstract:The neuroimaging method of near-infrared spectroscopy (NIRS) has gained traction in recent years, not least in Brain-Computer Interface (BCI) research. The higher spatial resolution compared to EEG and the lower price and portability compared to MRI predestine NIRS for certain applications. Although some groups design and build their own imaging systems, most of the research is conducted with commercial systems from a small number of manufacturers. In this study we compare three commercial NIRS systems with a motor execution paradigm. Finger tapping was chosen, because we expect a comparably strong hemodynamic response in the motor cortex, which is covered by the channels of the used systems. To simplify the process of comparing other researchers’ systems with ours we processed the data with the freely available software NIRS-SPM. With each system we measured 10 healthy right-handed subjects. Subjects performed right hand and left hand finger tapping in trials of 10 s. The sequence of right and left trials was randomized. Each trial was succeeded by a rest period of randomized length (15-20 s). The three NIRS systems were a ETG-4000 from Hitachi Medical Systems, a DYNOT Large Capacity System from NIRx Medical Technologies and a FOIRE-3000 from Shimadzu Corporation. The systems vary in the number of sources and detectors and therefore in the number of channels. Other parameters like sampling frequency, wave lengths of the lasers and setting of optodes vary, too. Data analysis is performed with NIRS-SPM, a toolbox for SPM designed for the processing of spectroscopic data. Although certain processing steps are different among the three systems due to different hardware, the overall logic of the processing is the same. The data is detrended (WaveletMDL) and the general linear model is applied for statistical analysis. We discuss the differences in specificity, significance and stability among subjects for the three NIRS systems. Reference: Ye, J.C., Tak, S., Jang, K.E., Jung, J.W., Jang, J.D., 2009. NIRS-SPM: Statistical parametric mapping for nearinfrared spectroscopy. NeuroImage 44, 428-447. : Presentation Preference (Complete): No Preference Linking Group (Complete): None selected Nanosymposium Information (Complete): 12.05.2011 Oasis, The Online Abstract Submission ... abstractsonline.com/.../SubmitPrinterFr... 1/2sonline.com/.../SubmitPrinterFr... 1/2
Sufficient Canonical Correlation Analysis

Yiwen Guo,Xiaoqing Ding,Changsong Liu,Jing-Hao Xue

DOI: https://doi.org/10.1109/tip.2016.2551374

IF: 10.6

2016-01-01

IEEE Transactions on Image Processing

Abstract:Canonical correlation analysis (CCA) is an effective way to find two appropriate subspaces in which Pearson's correlation coefficients are maximized between projected random vectors. Due to its well-established theoretical support and relatively efficient computation, CCA is widely used as a joint dimension reduction tool and has been successfully applied to many image processing and computer vision tasks. However, as reported, the traditional CCA suffers from overfitting in many practical cases. In this paper, we propose sufficient CCA (S-CCA) to relieve CCA's overfitting problem, which is inspired by the theory of sufficient dimension reduction. The effectiveness of S-CCA is verified both theoretically and experimentally. Experimental results also demonstrate that our S-CCA outperforms some of CCA's popular extensions during the prediction phase, especially when severe overfitting occurs.
Sparse canonical correlation analysis for recognition

Huimin Zhang,Weifeng Liu,Zheng-Jun Zha

DOI: https://doi.org/10.1145/2808492.2808509

2015-01-01

Abstract:Canonical correlation analysis (CCA) is one promising feature extraction and subspace learning method for multivariate vectors by exploiting the correlation between two multidimensional varia-bles in a linear way. Hence CCA has been widely employed in many applications such as statistics, economics and signal pro-cessing. However, the traditional CCA may be difficult to interpret especially when the original variables are expected to involve only a few components. In this paper, we propose sparse canonical cor-relation analysis (SCCA) to overcome the above problem. SCCA can find a reasonable trade-off between statistical fidelity and in-terpretability. Furthermore, we use a generalized power method to optimize the proposed SCCA algorithm. And finally we conduct extensive experiments for recognition on several popular databases including UCI datasets and USAA dataset. Experimental results demonstrate that the proposed SCCA algorithm outperforms the traditional CCA algorithm.
Canonical Correlation Analysis based on Hilbert-Schmidt Independence Criterion and Centered Kernel Target Alignment.

Billy Chang,Uwe Krüger,Rafal Kustra,Junping Zhang

2013-01-01

Abstract:Canonical correlation analysis (CCA) is a well established technique for identifying linear relationships among two variable sets. Kernel CCA (KCCA) is the most notable nonlinear extension but it lacks interpretability and robustness against irrelevant features. The aim of this article is to introduce two nonlinear CCA extensions that rely on the recently proposed Hilbert-Schmidt independence criterion and the centered kernel target alignment. These extensions determine linear projections that provide maximally dependent projected data pairs. The paper demonstrates that the use of linear projections allows removing irrelevant features, whilst extracting combinations of strongly associated features. This is exemplified through a simulation and the analysis of recorded data that are available in the literature.
Canonical Correlation Analysis on Riemannian Manifolds and Its Applications

Hyunwoo J Kim,Nagesh Adluru,Barbara B Bendlin,Sterling C Johnson,Baba C Vemuri,Vikas Singh

DOI: https://doi.org/10.1007/978-3-319-10605-2_17

Abstract:Canonical correlation analysis (CCA) is a widely used statistical technique to capture correlations between two sets of multi-variate random variables and has found a multitude of applications in computer vision, medical imaging and machine learning. The classical formulation assumes that the data live in a pair of vector spaces which makes its use in certain important scientific domains problematic. For instance, the set of symmetric positive definite matrices (SPD), rotations and probability distributions, all belong to certain curved Riemannian manifolds where vector-space operations are in general not applicable. Analyzing the space of such data via the classical versions of inference models is rather sub-optimal. But perhaps more importantly, since the algorithms do not respect the underlying geometry of the data space, it is hard to provide statistical guarantees (if any) on the results. Using the space of SPD matrices as a concrete example, this paper gives a principled generalization of the well known CCA to the Riemannian setting. Our CCA algorithm operates on the product Riemannian manifold representing SPD matrix-valued fields to identify meaningful statistical relationships on the product Riemannian manifold. As a proof of principle, we present results on an Alzheimer's disease (AD) study where the analysis task involves identifying correlations across diffusion tensor images (DTI) and Cauchy deformation tensor fields derived from T1-weighted magnetic resonance (MR) images.
Sparse Weighted Canonical Correlation Analysis

Wenwen Min,Juan Liu,Shihua Zhang

DOI: https://doi.org/10.48550/arXiv.1710.04792

2017-10-13

Abstract:Given two data matrices $X$ and $Y$, sparse canonical correlation analysis (SCCA) is to seek two sparse canonical vectors $u$ and $v$ to maximize the correlation between $Xu$ and $Yv$. However, classical and sparse CCA models consider the contribution of all the samples of data matrices and thus cannot identify an underlying specific subset of samples. To this end, we propose a novel sparse weighted canonical correlation analysis (SWCCA), where weights are used for regularizing different samples. We solve the $L_0$-regularized SWCCA ($L_0$-SWCCA) using an alternating iterative algorithm. We apply $L_0$-SWCCA to synthetic data and real-world data to demonstrate its effectiveness and superiority compared to related methods. Lastly, we consider also SWCCA with different penalties like LASSO (Least absolute shrinkage and selection operator) and Group LASSO, and extend it for integrating more than three data matrices.

Machine Learning
Revisiting Deep Generalized Canonical Correlation Analysis

Paris A. Karakasis,Nicholas D. Sidiropoulos

DOI: https://doi.org/10.1109/TSP.2023.3333212

2023-12-21

Abstract:Canonical correlation analysis (CCA) is a classic statistical method for discovering latent co-variation that underpins two or more observed random vectors. Several extensions and variations of CCA have been proposed that have strengthened our capabilities in terms of revealing common random factors from multiview datasets. In this work, we first revisit the most recent deterministic extensions of deep CCA and highlight the strengths and limitations of these state-of-the-art methods. Some methods allow trivial solutions, while others can miss weak common factors. Others overload the problem by also seeking to reveal what is not common among the views -- i.e., the private components that are needed to fully reconstruct each view. The latter tends to overload the problem and its computational and sample complexities. Aiming to improve upon these limitations, we design a novel and efficient formulation that alleviates some of the current restrictions. The main idea is to model the private components as conditionally independent given the common ones, which enables the proposed compact formulation. In addition, we also provide a sufficient condition for identifying the common random factors. Judicious experiments with synthetic and real datasets showcase the validity of our claims and the effectiveness of the proposed approach.

Machine Learning,Signal Processing
A Local Method for Canonical Correlation Analysis.

Tengju Ye,Zhipeng Xie,Ang Li

DOI: https://doi.org/10.1007/978-3-319-25207-0_39

2015-01-01

Abstract:Canonical Correlation Analysis CCA is a standard statistical technique for finding linear projections of two arbitrary vectors that are maximally correlated. In complex situations, the linearity of CCA is not applicable. In this paper, we propose a novel local method for CCA to handle the non-linear situations.We aim to find a series of local linear projections instead of a single globe one. We evaluate the performance of our method and CCA on two real-world datasets. Our experiments show that local method outperforms original CCA in several realistic cross-modal multimedia retrieval tasks.
Canonical Principal Angles Correlation Analysis for Two-View Data

Sheng Wang,Jianfeng Lu,Xingjian Gu,Chunhua Shen,Rui Xia,Jingyu Yang

DOI: https://doi.org/10.1016/j.jvcir.2015.12.001

IF: 2.887

2016-01-01

Journal of Visual Communication and Image Representation

Abstract:Canonical correlation analysis (CCA) is a popular method that has been widely used in information fusion. However, CCA requires that the data from two views must be paired, which is hard to satisfy in the real applications, moreover, it only considers the correlated information of the paired data. Thus, it cannot be used when there are only a little paired data or no paired data. In this paper, we propose a novel method named Canonical Principal Angles Correlation Analysis (CPACA) which does not need paired data during training stage. It makes classic CCA escape from the limitation of paired information. Its objective function can be constructed as follows: First, the correlation of two views is represented by the similarity between two subspace spanned by the principal components, which makes CPACA favorably compare with CCA in the case of limited paired data; Second, in order to increase the discriminative information of CPACA, we utilize manifold regularization to exploit the geometry of the marginal distribution. To optimize the objective function, we propose a new method to calculate the projected vectors. The experimental results show that the performance of CPACA is superior to that of traditional CCA and its variants. (C) 2015 Elsevier Inc. All rights reserved.
An Inexact Manifold Augmented Lagrangian Method for Adaptive Sparse Canonical Correlation Analysis with Trace Lasso Regularization

Kangkang Deng,Zheng Peng

DOI: https://doi.org/10.48550/arXiv.2003.09195

2020-03-20

Abstract:Canonical correlation analysis (CCA for short) describes the relationship between two sets of variables by finding some linear combinations of these variables that maximizing the correlation coefficient. However, in high-dimensional settings where the number of variables exceeds sample size, or in the case of that the variables are highly correlated, the traditional CCA is no longer appropriate. In this paper, an adaptive sparse version of CCA (ASCCA for short) is proposed by using the trace Lasso regularization. The proposed ASCCA reduces the instability of the estimator when the covariates are highly correlated, and thus improves its interpretation. The ASCCA is further reformulated to an optimization problem on Riemannian manifolds, and an manifold inexact augmented Lagrangian method is then proposed for the resulting optimization problem. The performance of the ASCCA is compared with the other sparse CCA techniques in different simulation settings, which illustrates that the ASCCA is feasible and efficient.

Optimization and Control,Machine Learning
High-Dimensional Canonical Correlation Analysis

Anna Bykhovskaya,Vadim Gorin

DOI: https://doi.org/10.48550/arXiv.2306.16393

2023-08-21

Abstract:This paper studies high-dimensional canonical correlation analysis (CCA) with an emphasis on the vectors that define canonical variables. The paper shows that when two dimensions of data grow to infinity jointly and proportionally, the classical CCA procedure for estimating those vectors fails to deliver a consistent estimate. This provides the first result on the impossibility of identification of canonical variables in the CCA procedure when all dimensions are large. As a countermeasure, the paper derives the magnitude of the estimation error, which can be used in practice to assess the precision of CCA estimates. Applications of the results to cyclical vs. non-cyclical stocks and to a limestone grassland data set are provided.

Econometrics,Probability,Statistics Theory

Reliability of carpal angle determinations.

Regularized Canonical Correlation Analysis with Unlabeled Data

Sparse Generalized Canonical Correlation Analysis: Distributed Alternating Iteration Based Approach.

Regularised Canonical Correlation Analysis: graphical lasso, biplots and beyond

Sparse Canonical Correlation Analysis: New Formulation and Algorithm.

Unconstrained Stochastic CCA: Unifying Multiview and Self-Supervised Learning

A Survey on Canonical Correlation Analysis

Sparse Kernel Canonical Correlation Analysis Via $\Ell_1$-Regularization

On Sparse Canonical Correlation Analysis

Canonical Correlation Analysis as Reduced Rank Regression in High Dimensions

Comparison of NIRS systems with a motor execution paradigm

Sufficient Canonical Correlation Analysis

Sparse canonical correlation analysis for recognition

Canonical Correlation Analysis based on Hilbert-Schmidt Independence Criterion and Centered Kernel Target Alignment.

Canonical Correlation Analysis on Riemannian Manifolds and Its Applications

Sparse Weighted Canonical Correlation Analysis

Revisiting Deep Generalized Canonical Correlation Analysis

A Local Method for Canonical Correlation Analysis.

Canonical Principal Angles Correlation Analysis for Two-View Data

An Inexact Manifold Augmented Lagrangian Method for Adaptive Sparse Canonical Correlation Analysis with Trace Lasso Regularization

High-Dimensional Canonical Correlation Analysis