Abstract: Fair principal component analysis (FPCA), a ubiquitous dimensionality reduction technique in signal processing and machine learning, aims to find a low-dimensional representation for a high-dimensional dataset in view of fairness. The FPCA problem is a non-convex and non-smooth optimization over the Stiefel manifold. The state-of-the-art methods for solving the problem are subgradient methods and semidefinite relaxation based methods. However, these two types of methods have their obvious limitations and thus are only suitable for efficiently solving the FPCA problem in very special scenarios. The goal of this paper is to develop efficient algorithms for solving the FPCA problem in general settings, especially the very high-dimensional setting. In this paper, we first transform the problem into a smooth non-convex concave minimax optimization over the Stiefel manifold. Then we propose an alternating Riemannian gradient (ARG) algorithm, which performs a Riemannian gradient descent step and an ordinary gradient projection step at each iteration, for solving the general non-convex concave minimax problems over Riemannian manifolds. We prove that ARG can find an $\varepsilon$-stationary point of the above problem within $O(\varepsilon^{-4})$ iterations. Simulation results show that, compared with the state-of-the-art methods, our proposed ARG algorithm can achieve better performance in terms of the solution quality and speed for solving the FPCA problems arising from signal processing and machine learning.

A Riemannian Geometry Underlying Stochastic Algorithm for Adaptive Principal Component Analysis

Robust Principal Component Analysis Based on Maximum Correntropy Criterion

Near-optimal stochastic approximation for online principal component estimation

An Alternating Riemannian Gradient Algorithm for Fair Principal Component Analysis

Diffusion approximations of Oja's online principal component analysis

An Extension of Fast Iterative Shrinkage‐thresholding Algorithm to Riemannian Optimization for Sparse Principal Component Analysis

Convergence of Oja's online principal component flow

Nearly optimal stochastic approximation for online principal subspace estimation

An Extension of Fast Iterative Shrinkage-thresholding to Riemannian Optimization for Sparse Principal Component Analysis

Variance Reduced Stochastic Optimization for PCA and PLS.

Robust Covariance Estimation for Distributed Principal Component Analysis

Stochastic Optimization for Kernel Pca

An Efficient Alternating Riemannian/Projected Gradient Descent Ascent Algorithm for Fair Principal Component Analysis

Robust Principal Component Analysis Via Joint Reconstruction and Projection.

Robust Principal Component Analysis: A Construction Error Minimization Perspective

Row-Sparse Principal Component Analysis via Coordinate Descent Method

Global convergence of Oja's PCA learning algorithm with a non-zero-approaching adaptive learning rate

On the optimality of the Oja's algorithm for online PCA

Fast Algorithms for Robust Principal Component Analysis with an Upper Bound on the Rank.

Randomized Method for Robust Principal Component Analysis.

An Alternating Manifold Proximal Gradient Method for Sparse PCA and Sparse CCA