Abstract:Factor analysis (FA) or principal component analysis (PCA) models the covariance matrix of the observed data as R = SS' + {\Sigma}, where SS' is the low-rank covariance matrix of the factors (aka latent variables) and {\Sigma} is the diagonal matrix of the noise. When the noise is anisotropic (aka nonuniform in the signal processing literature and heteroscedastic in the statistical literature), the diagonal elements of {\Sigma} cannot be assumed to be identical and they must be estimated jointly with the elements of SS'. The problem of estimating SS' and {\Sigma} in the above covariance model is the central theme of the present paper. After stating this problem in a more formal way, we review the main existing algorithms for solving it. We then go on to show that these algorithms have reliability issues (such as lack of convergence or convergence to infeasible solutions) and therefore they may not be the best possible choice for practical applications. Next we explain how to modify one of these algorithms to improve its convergence properties and we also introduce a new method that we call FAAN (Factor Analysis for Anisotropic Noise). FAAN is a coordinate descent algorithm that iteratively maximizes the normal likelihood function, which is easy to implement in a numerically efficient manner and has excellent convergence properties as illustrated by the numerical examples presented in the paper. Out of the many possible applications of FAAN we focus on the following two: direction-of-arrival (DOA) estimation using array signal processing techniques and portfolio selection for financial asset management.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is **how to estimate the low - rank covariance matrix \( \mathbf{S}\mathbf{S}^T \) and the diagonal noise matrix \( \boldsymbol{\Sigma} \) in factor analysis under anisotropic noise conditions**. Specifically, when the noise is non - uniform (i.e., anisotropic noise in the signal processing literature and heteroscedastic noise in the statistical literature), the diagonal elements of the noise matrix \( \boldsymbol{\Sigma} \) cannot be assumed to be the same and must be estimated together with the elements of \( \mathbf{S}\mathbf{S}^T \). ### Problem Background In factor analysis (FA) or principal component analysis (PCA), the covariance matrix \( \mathbf{R} \) of the observed data can be expressed as: \[ \mathbf{R} = \mathbf{S}\mathbf{S}^T+\boldsymbol{\Sigma} \] where \( \mathbf{S}\mathbf{S}^T \) is the low - rank covariance matrix of factors (or latent variables), and \( \boldsymbol{\Sigma} \) is the diagonal matrix of noise. When the noise is anisotropic, both \( \mathbf{S}\mathbf{S}^T \) and \( \boldsymbol{\Sigma} \) need to be estimated simultaneously. ### Problems of Existing Methods The paper reviews several existing algorithms and points out that these algorithms have reliability problems, such as lack of convergence or convergence to infeasible solutions. These problems make these algorithms may not be the best choice in practical applications. ### Proposed New Method To improve the convergence of existing algorithms, the paper proposes a new method, called **FAAN (Factor Analysis for Anisotropic Noise)**. FAAN is a coordinate descent algorithm that estimates \( \mathbf{S}\mathbf{S}^T \) and \( \boldsymbol{\Sigma} \) by iteratively maximizing the normal likelihood function. This method has the following characteristics: 1. **Easy to Implement**: Numerically efficient and easy to implement. 2. **Excellent Convergence**: Its good convergence performance has been verified by numerical experiments. ### Applications The paper focuses on discussing the performance of FAAN in two specific applications: 1. **Direction of Arrival (DOA) Estimation**: Using array signal processing techniques. 2. **Portfolio Selection**: Used for financial asset management. ### Main Contributions 1. **Improving Existing Algorithms**: Proposes an improved coordinate descent algorithm, which solves the convergence problems of existing algorithms. 2. **New Method FAAN**: Introduces a new method with excellent convergence performance and numerical stability. 3. **Theoretical Analysis**: Discusses the identifiability of the parameterized model and provides theoretical support. Through these contributions, the paper provides an effective and reliable solution for factor analysis under anisotropic noise conditions.

Low-rank covariance matrix estimation for factor analysis in anisotropic noise: application to array processing and portfolio selection

A Regularized High-Dimensional Positive Definite Covariance Estimator with High-Frequency Data

Large-Dimensional Positive Definite Covariance Estimation for High Frequency Data via Low-rank and Sparse Matrix Decomposition

On Minimum Trace Factor Analysis -- An Old Song Sung to a New Tune

Statistical Inference for High-Dimensional Matrix-Variate Factor Models

Statistical Inference for High-Dimensional Matrix-Variate Factor Model

Robust Factor Analysis Parameter Estimation

Knowing Factors or Factor Loadings, or Neither? Evaluating Estimators of Large Covariance Matrices with Noisy and Asynchronous Data

Principal Component Analysis versus Factor Analysis

Fat Tailed Factors

Computation of the Maximum Likelihood estimator in low-rank Factor Analysis

Anomaly component analysis

Deterministic parallel analysis: An improved method for selecting factors and principal components

Block-diagonal idiosyncratic covariance estimation in high-dimensional factor models for financial time series

A Toeplitz Covariance Matrix Reconstruction Approach for Direction-of-Arrival Estimation

A note on variational Bayesian factor analysis.

High-dimensional Factor Model and Its Applications to Statistical Machine Learning

Exponential Weighting and Random-Matrix-Theory-Based Filtering of Financial Covariance Matrices for Portfolio Optimization

Robust factored principal component analysis for matrix-valued outlier accommodation and detection