Abstract:Composite minimization is a powerful framework in large-scale convex optimization, based on decoupling of the objective function into terms with structurally different properties and allowing for more flexible algorithmic design. We introduce a new algorithmic framework for complementary composite minimization, where the objective function decouples into a (weakly) smooth and a uniformly convex term. This particular form of decoupling is pervasive in statistics and machine learning, due to its link to regularization. The main contributions of our work are summarized as follows. First, we introduce the problem of complementary composite minimization in general normed spaces; second, we provide a unified accelerated algorithmic framework to address broad classes of complementary composite minimization problems; and third, we prove that the algorithms resulting from our framework are near-optimal in most of the standard optimization settings. Additionally, we show that our algorithmic framework can be used to address the problem of making the gradients small in general normed spaces. As a concrete example, we obtain a nearly-optimal method for the standard $\ell_1$ setup (small gradients in the $\ell_{\infty}$ norm), essentially matching the bound of Nesterov (2012) that was previously known only for the Euclidean setup. Finally, we show that our composite methods are broadly applicable to a number of regression and other classes of optimization problems, where regularization plays a key role. Our methods lead to complexity bounds that are either new or match the best existing ones.

Gradient-prolongation commutativity and graph theory

Characterization of Smoothness of Multivariate Refinable Functions and Convergence of Cascade Algorithms of Nonhomogeneous Refinement Equations

A note on an application of discrete Morse theoretic techniques on the complex of disconnected graphs

Gradient flows on graphons: existence, convergence, continuity equations

Complex Gradient Systems

Gradients on Matrix Manifolds and their Chain Rule

Biharmonic Continuations of Gradients with the Help of Monogenic Functions with Values in the Biharmonic Algebra

Gradient descent for unbounded convex functions on Hadamard manifolds and its applications to scaling problems

A Geometric Understanding of Natural Gradient

A refined Lusin type theorem for gradients

Gradient systems on coupled cell networks

Gradation in Greyscales of Graphs

Generalised gradients for virtual elements and applications to a posteriori error analysis

Construction of a spanning tree for high‐order edge elements

From Combinatorics to Geometry: The Dynamics of Graph Gradient Diffusion

Gradient estimate for complex Monge-Ampere equation with continuous right hand side

A note on gradient-like vector fields

On the gradient rearrangement of functions

A geometric result for composite materials with $C^{1,γ}$-boundaries

Complementary Composite Minimization, Small Gradients in General Norms, and Applications

Spectral properties of the gradient operator with nonconstant coefficients