Abstract:This paper presents two new augmented flexible (AF)-Krylov subspace methods, AF-GMRES and AF-LSQR, to compute solutions of large-scale linear discrete ill-posed problems that can be modeled as the sum of two independent random variables, exhibiting smooth and sparse stochastic characteristics respectively. Following a Bayesian modelling approach, this corresponds to adding a covariance-weighted quadratic term and a sparsity enforcing l1 term in the original least-squares minimization scheme. To handle the l1 regularization term, the proposed approach constructs a sequence approximating quadratic problems that are partially solved using augmented flexible Krylov–Tikhonov methods. Compared to other traditional methods used to solve this minimization problem, such as those based on iteratively reweighted norm schemes, the new algorithms build a single (augmented, flexible) approximation (Krylov) subspace that encodes information about the different regularization terms through adaptable "preconditioning". The solution space is then expanded as soon as a new problem within the sequence is defined. This also allows for the regularization parameters to be chosen on-the-fly at each iteration. Compared to most recent work on generalized flexible Krylov methods [6] , our methods offer theoretical assurance of convergence and a more stable numerical performance. The efficiency of the new methods is shown through a variety of experiments, including a synthetic image deblurring problem, a synthetic atmospheric transport problem, and fluorescence molecular tomography reconstructions using both synthetic and real-world experimental data.

An Extension of Averaged-Operator-Based Algorithms

A variable smoothing for Nonconvexly constrained nonsmooth optimization with application to sparse spectral clustering

Computational approaches to non-convex, sparsity-inducing multi-penalty regularization

An alternating iterative minimisation algorithm for the double-regularised total least square functional

Separable Bregman Framework for Sparsity Constrained Nonlinear Optimization

Augmented Flexible Krylov Subspace methods with applications to Bayesian inverse problems

An efficient active-set method with applications to sparse approximations and risk minimization

Sparsity-Aware Adaptive Algorithms Based on Alternating Optimization with Shrinkage

Algorithm-agnostic low-rank approximation of operator monotone matrix functions

Exploiting Sparsity in the Coefficient Matching Conditions in Sum-of-Squares Programming Using ADMM

Black-box Optimization Algorithms for Regularized Least-squares Problems

Stochastic Variance-Reduced Majorization-Minimization Algorithms

Adaptive Methods or Variational Inequalities with Relatively Smooth and Reletively Strongly Monotone Operators

ADMM Algorithmic Regularization Paths for Sparse Statistical Machine Learning

Minimization Over the Nonconvex Sparsity Constraint Using A Hybrid First-order method

Lp quasi-norm minimization: algorithm and applications

Smoothing accelerated algorithm for constrained nonsmooth convex optimization problems

String-Averaging Projected Subgradient Methods for Constrained Minimization

Algorithms on the sparse solution of under-determined linear systems

A Single-Loop Stochastic Proximal Quasi-Newton Method for Large-Scale Nonsmooth Convex Optimization