Abstract:SIAM Journal on Imaging Sciences, Volume 17, Issue 2, Page 1040-1077, June 2024. We consider a class of optimization problems defined over trees with unary cost terms and shifted pairwise cost terms. These problems arise when considering block coordinate descent (BCD) approaches for solving inverse problems with total generalized variation (TGV) regularizers or their nonconvex generalizations. We introduce a linear-time reduction that transforms the shifted problems into their nonshifted counterparts. However, combining existing continuous dynamic programming (DP) algorithms with the reduction does not lead to BCD iterations that compute TGV-like solutions. This problem can be overcome by considering a box-constrained modification of the subproblems or smoothing the cost terms of the TGV regularized problem. The former leads to shifted and box-constrained subproblems, for which we propose a linear-time reduction to their unconstrained counterpart. The latter naturally leads to problems with smooth unary and pairwise cost terms. With this in mind, we propose two novel continuous DP algorithms that can solve (convex and nonconvex) problems with piecewise quadratic unary and pairwise cost terms. We prove that the algorithm for the convex case has quadratic worst-case time and memory complexity, while the algorithm for the nonconvex case has exponential time and memory complexity, but works well in practice for smooth truncated total variation pairwise costs. Finally, we demonstrate the applicability of the proposed algorithms for solving inverse problems with first-order and higher-order regularizers.

An active-set based recursive approach for solving convex isotonic regression with generalized order restrictions

A Genetic Algorithm for Multivariate Isotonic Regression

On the Correctness of the Generalized Isotonic Recursive Partitioning Algorithm

SVD-free Convex-Concave Approaches for Nuclear Norm Regularization.

Total Generalized Variation on a Tree

A Smoothing Proximal Gradient Algorithm for Nonsmooth Convex Regression with Cardinality Penalty.

Decomposable Non-Smooth Convex Optimization with Nearly-Linear Gradient Oracle Complexity

Mixed-Integer Convex Nonlinear Optimization with Gradient-Boosted Trees Embedded

A SUCCESSIVE CONVEX APPROXIMATION APPROACH FOR SPARSE SOLUTIONS OF CONVEX PROGRAMS

A Unified Framework for Solving a General Class of Nonconvexly Regularized Convex Models

A General Iterative Shrinkage and Thresholding Algorithm for Non-convex Regularized Optimization Problems

Convex Relaxation Regression: Black-Box Optimization of Smooth Functions by Learning Their Convex Envelopes

Nonconvex Sparse Logistic Regression with Weakly Convex Regularization

Optimization of Graph Total Variation via Active-Set-based Combinatorial Reconditioning

Actively Learning Combinatorial Optimization Using a Membership Oracle

ROOT-SGD: Sharp Nonasymptotics and Near-Optimal Asymptotics in a Single Algorithm

On Regularized Square-root Regression Problems: Distributionally Robust Interpretation and Fast Computations

Modified inertial subgradient extragradient algorithms for generalized equilibria systems with constraints of variational inequalities and fixed points

Improved Optimization Methods For Regularized Optimal Transport

An accelerated active-set algorithm for a quadratic semidefinite program with general constraints

An efficient nonconvex reformulation of stagewise convex optimization problems