Abstract:For many applications in signal processing and machine learning, we are tasked with minimizing a large sum of convex functions subject to a large number of convex constraints. In this paper, we devise a new random projection method (RPM) to efficiently solve this problem. Compared with existing RPMs, our proposed algorithm features two useful algorithmic ideas. First, at each iteration, instead of projecting onto the subset defined by one of the constraints, our algorithm only requires projecting onto a half-space approximation of the subset, which significantly reduces the computational cost as it admits a closed-form formula. Second, to exploit the structure that the objective is a sum, variance reduction is incorporated into our algorithm to further improve the performance. As theoretical contributions, under a novel error bound condition and other standard assumptions, we prove that the proposed RPM converges to an optimal solution and that both optimality and feasibility gaps vanish at a sublinear rate. In particular, via a new analysis framework, we show that our RPM attains a faster convergence rate in optimality gap than existing RPMs when the objective function has a Lipschitz continuous gradient, capitalizing the benefit of the variance reduction. We also provide sufficient conditions for the error bound condition to hold. Experiments on a beamforming problem and a robust classification problem are also presented to demonstrate the superiority of our RPM over existing ones.

Optimal truncation for random process in prespecified coordinate

Quality of Approximate Balanced Truncation

Stability and error bounds for discrete time frequency weighted balanced truncation

The TRUNC element in any dimension and application to a modified Poisson equation

Information theoretic methods for stochastic model reduction based on state projection

Minimizing Sum of Truncated Convex Functions and Its Applications

A Minimum Output Information Loss Method for Stochastic Model Reduction

Convergence of Trimmed Lévy Processes to Trimmed Stable Random Variables at $0$

First Order Reliability Method With Truncated Random Variables

Quantitative truncation estimates for fractional Hardy-Sobolev optimizers

Projection-free approximate balanced truncation of large unstable systems

Optimal sub-Gaussian variance proxy for truncated Gaussian and exponential random variables

Low-rank balanced truncation of discrete time-delay systems based on Laguerre expansions

An Optimal Spatial Finite-Difference Operator Which Reduces Truncation Error to a Minimum

Balanced truncation for linear switched systems

Balanced truncation with relative/multiplicative error bounds in L ∞ norm

Balanced Truncation Model Reduction of Nonstationary Systems Interconnected over Arbitrary Graphs

Determining the Optimum Process Mean for Truncated Normal Distribution

Edge limits of truncated circular beta ensembles

Variance Reduced Random Relaxed Projection Method for Constrained Finite-sum Minimization Problems