Abstract:For many applications in signal processing and machine learning, we are tasked with minimizing a large sum of convex functions subject to a large number of convex constraints. In this paper, we devise a new random projection method (RPM) to efficiently solve this problem. Compared with existing RPMs, our proposed algorithm features two useful algorithmic ideas. First, at each iteration, instead of projecting onto the set defined by one of the constraints, our algorithm only requires projecting onto a half-space approximation of the set, which significantly reduces the computational cost as it admits a closed-form formula. Second, to exploit the structure that the objective is a sum, variance reduction is incorporated into our algorithm to further improve the performance. As theoretical contributions, under a novel error bound condition and other standard assumptions, we prove that the proposed RPM converges to an optimal solution and that both optimality and feasibility gaps vanish at a sublinear rate. In particular, via a new analysis framework, we show that our RPM attains a faster convergence rate in optimality gap than existing RPMs when the objective function has a Lipschitz continuous gradient, capitalizing the benefit of the variance reduction. We also provide sufficient conditions for the error bound condition to hold. Experiments on a beamforming problem and a robust classification problem are presented to demonstrate the superiority of our RPM over existing ones.

The Relaxed Inexact Projection Methods for the Split Feasibility Problem

A GLOBALLY AND SUPERLIEARLY CONVERGENT FEASIBLE SQP ALGORITHM FOR DEGENERATE CONSTRAINED OPTIMIZATION

On Descent-Projection Method for Solving the Split Feasibility Problems

Strong convergence of Bregman projection algorithms for solving split feasibility problems

Self-adaptive relaxed CQ algorithms for solving split feasibility problem with multiple output sets

An inertial hyperplane projection method for split feasibility problems and applications

Linear Regularity and Linear Convergence of Projection-Based Methods for Solving Convex Feasibility Problems

On inertial non-lipschitz stepsize algorithms for split feasibility problems

PROJECTION ALGORITHMS FOR CONVEX FEASIBILITY PROBLEMS ON HADAMARD MANIFOLDS

An inertial accelerated outer quadratic approximation method for split feasibility problem with application to elastic net

Two Relaxed CQ Methods for the Split Feasibility Problem with Multiple Output Sets

Subgradient Projection Algorithms for Convex Feasibility on Riemannian Manifolds with Lower Bounded Curvatures

Relative Regularity Conditions and Linear Regularity Properties for Split Feasibility Problems in Normed Linear Spaces

An improved relaxed inertial projection algorithm for solving the minimum-norm solution of variational inequality and fixed point problems

Algorithms and Convergence Results of Projection Methods for Inconsistent Feasibility Problems: A Review

On inexact stochastic splitting methods for a class of nonconvex composite optimization problems with relative error

Projection Methods: Swiss Army Knives for Solving Feasibility and Best Approximation Problems with Halfspaces

Extragradient method with feasible inexact projection to variational inequality problem

Inertial Halpern-type iterative algorithm for the generalized multiple-set split feasibility problem in Banach spaces

A Relaxed Tseng's Method-based Algorithm for Solving Split Variational Inequalities with Multiple Output Sets

Variance Reduced Random Relaxed Projection Method for Constrained Finite-Sum Minimization Problems