Abstract:Factor models have been widely used to summarize the variability of high-dimensional data through a set of factors with much lower dimensionality. Gaussian linear factor models have been particularly popular due to their interpretability and ease of computation. However, in practice, data often violate the multivariate Gaussian assumption. To characterize higher-order dependence and nonlinearity, models that include factors as predictors in flexible multivariate regression are popular, with GP-LVMs using Gaussian process (GP) priors for the regression function and VAEs using deep neural networks. Unfortunately, such approaches lack identifiability and interpretability and tend to produce brittle and non-reproducible results. To address these problems by simplifying the nonparametric factor model while maintaining flexibility, we propose the NIFTY framework, which parsimoniously transforms uniform latent variables using one-dimensional nonlinear mappings and then applies a linear generative model. The induced multivariate distribution falls into a flexible class while maintaining simple computation and interpretation. We prove that this model is identifiable and empirically study NIFTY using simulated data, observing good performance in density estimation and data visualization. We then apply NIFTY to bird song data in an environmental monitoring application.

What problem does this paper attempt to address?

The main issues that this paper attempts to address with existing factor models when dealing with high-dimensional data are: 1. **Non-normality**: Many real-world data do not conform to the multivariate normal distribution assumption, causing traditional Gaussian linear factor models (as shown in Equation (2)) to perform poorly in these cases. For example, when data exhibit skewed or bimodal distributions, traditional models fail to capture these characteristics effectively. 2. **Nonlinear dependencies**: Real-world data often contain complex nonlinear relationships, which traditional linear factor models cannot effectively capture. For instance, Figure 1 illustrates the failure of traditional models in handling non-Gaussian marginal distributions and nonlinear dependencies. 3. **Identifiability and interpretability**: Existing nonlinear factor models (such as Variational Autoencoders (VAE) and Gaussian Process Latent Variable Models (GP-LVM)) are flexible but lack identifiability and interpretability, leading to unstable and hard-to-reproduce results. To address these issues, the authors propose a new framework called NIFTY (Nonparametric Identifiable and Flexible Transformations for Yields). The NIFTY framework simplifies nonparametric factor models while maintaining flexibility through the following methods: - **Nonparametric transformations**: NIFTY uses one-dimensional nonlinear mappings to transform uniform latent variables into low-dimensional factors, followed by applying a linear generative model. This transformation method allows the model to capture complex nonlinear relationships while maintaining simple computation and interpretability. - **Strict identifiability**: The authors demonstrate that the NIFTY model is strictly identifiable under mild assumptions, meaning there is a one-to-one correspondence between model parameters and observed data, ensuring the model's reliability and reproducibility. - **Distribution constraints**: NIFTY introduces a distribution constraint method that addresses the posterior distribution drift problem by restricting the empirical distribution of latent variables to be close to a uniform distribution, thereby improving the model's stability and accuracy. In summary, the NIFTY framework aims to address the limitations of existing factor models in handling high-dimensional, non-normal, and nonlinear data by simplifying nonparametric factor models while maintaining their flexibility and interpretability.

Identifiable and interpretable nonparametric factor analysis

Nonnegative spatial factorization

Identifiability of latent-variable and structural-equation models: from linear to nonlinear

Dependent Indian Buffet Process-based Sparse Nonparametric Nonnegative Matrix Factorization

Factor modelling for high-dimensional functional time series

Expandable Factor Analysis

Optimal Estimation of Large-Dimensional Nonlinear Factor Models

Infinite Mixtures of Infinite Factor Analysers

Nonparametric Bayesian Nonnegative Matrix Factorization

Factor Models for High‐Dimensional Functional Time Series I: Representation Results

Investigations on Non-Gaussian Factor Analysis

A nonparametric variable clustering model

Exponential Family Factors for Bayesian Factor Analysis

Non-negative Independent Factor Analysis disentangles discrete and continuous sources of variation in scRNA-seq data

Fat Tailed Factors

High-Dimensional Overdispersed Generalized Factor Model With Application to Single-Cell Sequencing Data Analysis

High-dimensional Factor Model and Its Applications to Statistical Machine Learning

High-dimensional Factor Analysis for Network-linked Data

Modeling High-Dimensional Time Series: A Factor Model With Dynamically Dependent Factors and Diverging Eigenvalues

The Kernel Trick for Nonlinear Factor Modeling

Factor Models for High‐Dimensional Functional Time Series II: Estimation and Forecasting