Abstract:This paper is a follow up to the previous author's paper on convex optimization. In that paper we began the process of adjusting greedy-type algorithms from nonlinear approximation for finding sparse solutions of convex optimization problems. We modified there three the most popular in nonlinear approximation in Banach spaces greedy algorithms -- Weak Chebyshev Greedy Algorithm, Weak Greedy Algorithm with Free Relaxation and Weak Relaxed Greedy Algorithm -- for solving convex optimization problems. We continue to study sparse approximate solutions to convex optimization problems. It is known that in many engineering applications researchers are interested in an approximate solution of an optimization problem as a linear combination of elements from a given system of elements. There is an increasing interest in building such sparse approximate solutions using different greedy-type algorithms. In this paper we concentrate on greedy algorithms that provide expansions, which means that the approximant at the $m$th iteration is equal to the sum of the approximant from the previous iteration ($(m-1)$th iteration) and one element from the dictionary with an appropriate coefficient. The problem of greedy expansions of elements of a Banach space is well studied in nonlinear approximation theory. At a first glance the setting of a problem of expansion of a given element and the setting of the problem of expansion in an optimization problem are very different. However, it turns out that the same technique can be used for solving both problems. We show how the technique developed in nonlinear approximation theory, in particular, the greedy expansions technique can be adjusted for finding a sparse solution of an optimization problem given by an expansion with respect to a given dictionary.

Greedy Approximation in Convex Optimization

Greedy expansions in convex optimization

Optimization of Approximate Solution of Integral Equations of Several Variables

Convergence and rate of convergence of some greedy algorithms in convex optimization

Sparse approximation by greedy algorithms

Biorthogonal Greedy Algorithms in convex optimization

On greedy approximation in complex Banach spaces

Linear Convergence of Stochastic Iterative Greedy Algorithms With Sparse Constraints

Greed is Good: Algorithmic Results for Sparse Approximation

Forward Basis Selection for Sparse Approximation over Dictionary

Greedy Sparsity-Constrained Optimization

On the rate of convergence of greedy algorithms

On Some Versions of Subspace Optimization Methods with Inexact Gradient Information

DC approximation approaches for sparse optimization

A SUCCESSIVE CONVEX APPROXIMATION APPROACH FOR SPARSE SOLUTIONS OF CONVEX PROGRAMS

Optimal and parameter-free gradient minimization methods for convex and nonconvex optimization

Newton Greedy Pursuit: A Quadratic Approximation Method for Sparsity-Constrained Optimization.

Efficient and provably convergent randomized greedy algorithms for neural network optimization

Construction of Optimal Algorithms for Function Approximation in Gaussian Sobolev Spaces

An efficient active-set method with applications to sparse approximations and risk minimization

Approximating optimal solutions to biconvex parametric programs