Abstract: The classical non-greedy algorithm (NGA) and the recently proposed proximal alternating minimization method with extrapolation (PAMe) for $L_1$-norm PCA are revisited and their finite-step convergence are studied. It is first shown that NGA can be interpreted as a conditional subgradient or an alternating maximization method. By recognizing it as a conditional subgradient, we prove that the iterative points generated by the algorithm will be constant in finitely many steps under a certain full-rank assumption; such an assumption can be removed when the projection dimension is one. By treating the algorithm as an alternating maximization, we then prove that the objective value will be fixed after at most $\left\lceil\frac{F^{\max}}{\tau_0} \right\rceil$ steps, where the stopping point satisfies certain optimality conditions. Then, a slight modification of NGA with improved convergence properties is analyzed. It is shown that the iterative points generated by the modified algorithm will not change after at most $\left\lceil\frac{2F^{\max}}{\tau} \right\rceil$ steps; furthermore, the stopping point satisfies certain optimality conditions if the proximal parameter $\tau$ is small enough. For PAMe, it is proved that the sign variable will remain constant after finitely many steps and the algorithm can output a point satisfying certain optimality condition, if the parameters are small enough and a full rank assumption is satisfied. Moreover, if there is no proximal term on the projection matrix related subproblem, then the iterative points generated by this modified algorithm will not change after at most $\left\lceil \frac{4F^{\max}}{\tau(1-\gamma)} \right\rceil$ steps and the stopping point also satisfies certain optimality conditions, provided similar assumptions as those for PAMe. The full rank assumption can be removed when the projection dimension is one.

Projected Policy Gradient Converges in a Finite Number of Iterations

On the Convergence of Projected Policy Gradient for Any Constant Step Sizes

On the Sublinear Convergence of Projected Policy Gradient for Any Constant Step Sizes

Elementary Analysis of Policy Gradient Methods

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Convergence of Policy Gradient Methods for Finite-Horizon Exploratory Linear-Quadratic Control Problems

On the Linear Convergence of Natural Policy Gradient Algorithm

Convergence of Policy Gradient for Stochastic Linear-Quadratic Control Problem in Infinite Horizon

Stochastic Cubic-Regularized Policy Gradient Method

Linear Convergence for Natural Policy Gradient with Log-linear Policy Parametrization

Convex Optimization Via Finite-Time Projected Gradient Flows

Convergence rate analysis of distributed optimization with projected subgradient algorithm.

Projective Proximal Gradient Descent for A Class of Nonconvex Nonsmooth Optimization Problems: Fast Convergence Without Kurdyka-Lojasiewicz (KL) Property

On the Linear Convergence of Policy Gradient under Hadamard Parameterization

Proximal-Proximal-Gradient Method

On Finite-Step Convergence of the Non-Greedy Algorithm and Proximal Alternating Minimization Method with Extrapolation for $L_1$-Norm PCA

Convergence and sample complexity of natural policy gradient primal-dual methods for constrained MDPs

Convergence Rate of Projected Subgradient Method with Time-varying Step-sizes

Gradient descent with adaptive stepsize converges (nearly) linearly under fourth-order growth

A Conjugate Gradient Algorithm with Function Value Information and N-Step Quadratic Convergence for Unconstrained Optimization.

GLOBAL CONVERGENCE OF A MODIFIED PRP CONJUGATE GRADIENT METHOD AND ITS NUMERICAL RESULTS