Abstract:Factor analysis, often regarded as a Bayesian variant of matrix factorization, offers superior capabilities in capturing uncertainty, modeling complex dependencies, and ensuring robustness. As the deep learning era arrives, factor analysis is receiving less and less attention due to their limited expressive ability. On the contrary, contrastive learning has emerged as a potent technique with demonstrated efficacy in unsupervised representational learning. While the two methods are different paradigms, recent theoretical analysis has revealed the mathematical equivalence between contrastive learning and matrix factorization, providing a potential possibility for factor analysis combined with contrastive learning. Motivated by the interconnectedness of contrastive learning, matrix factorization, and factor analysis, this paper introduces a novel Contrastive Factor Analysis framework, aiming to leverage factor analysis's advantageous properties within the realm of contrastive learning. To further leverage the interpretability properties of non-negative factor analysis, which can learn disentangled representations, contrastive factor analysis is extended to a non-negative version. Finally, extensive experimental validation showcases the efficacy of the proposed contrastive (non-negative) factor analysis methodology across multiple key properties, including expressiveness, robustness, interpretability, and accurate uncertainty estimation.

The Contrast Algorithm for Unbalanced Multifactor Analysis of Variance

Sparse matrix methods for unbalanced multifactor analysis of variance and covariance

A note on extending Scheffé's modified multiple-comparison procedure to other analysis situations

Quadratic Form based Multiple Contrast Tests for Comparison of Group Means

A theory of contrasts for modified Freeman–Tukey statistics and its applications to Tukey's post-hoc tests for contingency tables

Fitting Additive Models to Unbalanced Two-Way Data

A Unified Framework for Contrast Research of the Latent Variable Multivariate Regression Methods

Interaction Balance for Symmetrical Factorial Designs with Generalized Minimum Aberration

Comparing Coefficients of Nested Nonlinear Probability Models

Multiple Comparisons of Treatment Against Control under Unequal Variances Using Parametric Bootstrap

Computer Programs for Robust Analyses in Multifactor Analysis of Variance Designs

Contrastive Factor Analysis

A Multiple Comparisons Procedure for Detecting Differences Between Treatments and A Control in Two-Factor Experiments

New multivariate tests for assessing covariate balance in matched observational studies

An Aligned Rank Transform Procedure for Multifactor Contrast Tests

A comparison of recent approaches to the analysis of repeated measurements

A compromise criterion for weighted least squares estimates

Critical Analysis of Factorial Experiments and Orthogonal Fractions

Optimal experimental designs for treatment contrasts in heteroscedastic models with covariates

Comment: the Essential Tension Between Parsimony and Accuracy

A more powerful test method for analyzing unreplicated factorial two-level experiments