Convex Regularization of Multi-Channel Images Based on Variants of the TV-Model

Martin Fuchs,Jan Mueller,Christian Tietz,Joachim Weickert

DOI: https://doi.org/10.1080/17476933.2017.1386181

2018-04-04

Abstract:We discuss existence and regularity results for multi-channel images in the setting of isotropic and anisotropic variants of the TV-model.

Analysis of PDEs

What problem does this paper attempt to address?

Convex and non-convex adaptive TV regularizations for color image restoration

Xinv Wang,Mingxi Ma,Jingjing Lu,Jun Zhang

DOI: https://doi.org/10.1007/s40314-023-02552-y

2024-01-03

Computational and Applied Mathematics

Abstract:Color image restoration is an important and challenging research topic in image processing. Different from grayscale images, each color image has three channels in RGB color space. Due to the correlation among the three channels, color total variation (TV) regularized image restoration based on the local channel coupling is better than the direct application of its grayscale counterpart in each channel of color images. On the other hand, an adaptive weighting scheme is a good technique for restoring local features of images. Inspired by these two strategies, we propose convex and non-convex adaptive TV regularized models for color image restoration to better handle image local features. Numerically, we design an alternating direction method of multipliers to efficiently solve the proposed two models. Comprehensive experiments are conducted to demonstrate the effectiveness and advantages of the proposed methods.

mathematics, applied
Universal Minimality in TV Regularization with Rectilinear Anisotropy

Clemens Kirisits,Eric Setterqvist

DOI: https://doi.org/10.48550/arXiv.1910.05186

2019-10-11

Optimization and Control

Abstract:In this article, we study theoretical aspects of the $\ell^{1}$-anisotropic Rudin-Osher-Fatemi (ROF) model on hyperrectangular domains of arbitrary dimension. Our main result states that the solution of the ROF model minimizes a large class of convex functionals, including all $L^p$-norms, over a certain neighbourhood of the datum. This neighbourhood arises naturally in the dual formulation and is given by the TV subdifferential at zero, scaled by the regularization parameter and translated by the datum. In order to prove the main result, we exploit the recently established fact that, for piecewise constant datum, the ROF model reduces to a finite-dimensional problem. We then show a finite-dimensional version of the main result, before translating it back to the original setting and finishing the proof by means of an approximation argument.
Identifiability of Multichannel Blind Deconvolution and Nonconvex Regularization Algorithm

Yu Xia,Song Li

DOI: https://doi.org/10.1109/tsp.2018.2866826

IF: 4.875

2018-01-01

IEEE Transactions on Signal Processing

Abstract:In this paper, we consider the multichannel blind deconvolution problem, where we observe the output of channels $\mathbf {h}_{i}\in \mathbb {R}^n$( $i=1,...,N$) that all convolve with the same unknown input signal $\mathbf {x}\in \mathbb {R}^n$ . We wish to estimate the input signal and blur kernels simultaneously. Existing theoretical results showed that the original inputs are identifiable under subspace assumptions. However, the subspaces discussed before were randomly or generically chosen. Here, we propose deterministic subspace assumption, which is widely used in practice, and give some theoretical results. First of all, we derive tight sufficient condition for identifiability of signal and convolution kernels, which is only violated on a set of Lebesgue measure zero. Then, we present a nonconvex regularization algorithm by a lifting method and approximate the rank-one constraint via the difference of nuclear norm and Frobenius norm. The global minimizer of the proposed nonconvex algorithm is rank-one matrix under mild conditions on parameters and noise level. The stability result is also shown under the assumption that the inputs lie in a compact set. Besides, the computation of our regularization model is carried out and any limit point of iterations converges to a stationary point of our model. Finally, we provide numerical experiments to show that our nonconvex regularization model outperforms convex relaxation models, such as nuclear norm minimization and some nonconvex methods, such as alternating minimization method and spectral method.
Regularization with Sparse Vector Fields: From Image Compression to TV-type Reconstruction

Eva-Maria Brinkmann,Martin Burger,Joana Grah

DOI: https://doi.org/10.48550/arXiv.1503.02044

2015-03-03

Optimization and Control

Abstract:This paper introduces a novel variational approach for image compression motivated by recent PDE-based approaches combining edge detection and Laplacian inpainting. The essential feature is to encode the image via a sparse vector field, ideally concentrating on a set of measure zero. An equivalent reformulation of the compression approach leads to a variational model resembling the ROF-model for image denoising, hence we further study the properties of the effective regularization functional introduced by the novel approach and discuss similarities to TV and TGV functionals. Moreover we computationally investigate the behaviour of the model with sparse vector fields for compression in particular for high resolution images and give an outlook towards denoising.
Non-convex regularization based on shrinkage penalty function

Manu Ghulyani,Muthuvel Arigovindan

2023-09-09

Abstract:Total Variation regularization (TV) is a seminal approach for image recovery. TV involves the norm of the image's gradient, aggregated over all pixel locations. Therefore, TV leads to piece-wise constant solutions, resulting in what is known as the "staircase effect." To mitigate this effect, the Hessian Schatten norm regularization (HSN) employs second-order derivatives, represented by the pth norm of eigenvalues in the image hessian, summed across all pixels. HSN demonstrates superior structure-preserving properties compared to TV. However, HSN solutions tend to be overly smoothed. To address this, we introduce a non-convex shrinkage penalty applied to the Hessian's eigenvalues, deviating from the convex lp norm. It is important to note that the shrinkage penalty is not defined directly in closed form, but specified indirectly through its proximal operation. This makes constructing a provably convergent algorithm difficult as the singular values are also defined through a non-linear operation. However, we were able to derive a provably convergent algorithm using proximal operations. We prove the convergence by establishing that the proposed regularization adheres to restricted proximal regularity. The images recovered by this regularization were sharper than the convex counterparts.

Optimization and Control,Image and Video Processing
Fast Optimization for Multichannel Total Variation Minimization with Non-Quadratic Fidelity

Juyong Zhang,Chunlin Wu

DOI: https://doi.org/10.1016/j.sigpro.2011.02.015

IF: 4.729

2011-01-01

Signal Processing

Abstract:Total variation (TV) has been proved very successful in image processing, and it has been combined with various non-quadratic fidelities for non-Gaussian noise removal. However, these models are hard to solve because TV is non-differentiable and nonlinear, and non-quadratic fidelity term is also nonlinear and even non-differentiable for some special cases. This prevents their widespread use in practical applications. Very recently, it was found that the augmented Lagrangian method is extremely efficient for this kind of models. However, only the single-channel case (e.g., gray images) is considered. In this paper, we propose a general computational framework based on augmented Lagrangian method for multichannel TV minimization with non-quadratic fidelity, and then show how to apply it to two special cases: L1 and Kullback–Leibler (KL) fidelities, two common and important data terms for blurry images corrupted by impulsive noise or Poisson noise, respectively. For these typical fidelities, we show that the sub-problems either can be fast solved by FFT or have closed form solutions. The experiments demonstrate that our algorithm can fast restore high quality images.
A Tensor-Based Nonlocal Total Variation Model for Multi-Channel Image Recovery

Wenfei Cao,Jing Yao,Jian Sun,Guodong Han

DOI: https://doi.org/10.1016/j.sigpro.2018.07.019

IF: 4.729

2018-01-01

Signal Processing

Abstract:In this paper, a new nonlocal total variation (NLTV) regularizer is proposed for solving the inverse problems in multi-channel image processing. Different from the existing nonlocal total variation regularizers that rely on the graph gradient, the proposed nonlocal total variation involves the standard image gradient and simultaneously exploits three important properties inherent in multi-channel images through a tensor nuclear norm, hence we call this proposed functional as tensor-based nonlocal total variation (TenNLTV). In specific, these three properties are the local structural image regularity, the nonlocal image self-similarity, and the image channel correlation, respectively. By fully utilizing these three properties, TenNLTV can provide a more robust measure of image variation. Then, based on the proposed regularizer TenNLTV, a novel regularization model for inverse imaging problems is presented. Moreover, an effective algorithm is designed for the proposed model, and a closed-form solution is derived for a two-order complex eigen system in our algorithm. Extensive experimental results on several inverse imaging problems demonstrate that the proposed regularizer is systematically superior over other competing local and nonlocal regularization approaches, both quantitatively and visually. (C) 2018 Elsevier B.V. All rights reserved.
Image Restoration by Combined Order Regularization with Optimal Spatial Adaptation

Sanjay Viswanath,Simon de Beco,Maxime Dahan,Muthuvel Arigovindan

DOI: https://doi.org/10.1109/TIP.2020.2988146

2021-06-01

Abstract:Total Variation (TV) and related extensions have been popular in image restoration due to their robust performance and wide applicability. While the original formulation is still relevant after two decades of extensive research, its extensions that combine derivatives of first- and second-order are now being explored for better performance, with examples being Combined Order TV (COTV) and Total Generalized Variation (TGV). As an improvement over such multi-order convex formulations, we propose a novel non-convex regularization functional which adaptively combines Hessian-Schatten (HS) norm and first order TV (TV1) functionals with spatially varying weight. This adaptive weight itself is controlled by another regularization term; the total cost becomes the sum of this adaptively weighted HS-TV1 term, the regularization term for the adaptive weight, and the data-fitting term. The reconstruction is obtained by jointly minimizing w.r.t. the required image and the adaptive weight. We construct a block coordinate descent method for this minimization with proof of convergence, which alternates between minimization w.r.t. the required image and the adaptive weights. We derive exact computational formula for minimization w.r.t. the adaptive weight, and construct an ADMM algorithm for minimization w.r.t. to the required image. We compare the proposed method using image recovery examples including MRI reconstruction and microscopy deconvolution.

Image and Video Processing
Image Restoration Via Multi-Scale Non-Local Total Variation Regularization

Jing Mu,Ruiqin Xiong,Xiaopeng Fan,Siwei Ma

DOI: https://doi.org/10.1109/icme.2017.8019463

2017-01-01

Abstract:Total-variation (TV) regularization is widely adopted in image restoration problems to exploit the local smoothness of image. However, traditional TV regularization only models the sparsity of image gradient at the original scale. This paper introduces a multi-scale TV regularization method which models the image gradient sparsity at different scales, and constrains the gradient magnitude of different scales jointly. As different scales extract different frequency of image, our proposed multi-scale regularization method provides constraints for different frequency components. And for each scale, we adaptively estimate the gradient distribution at a particular pixel from a group of nonlocally searched similar patches. Finally, experimental results demonstrate that the proposed method outperforms the conventional TV regularization methods for image restoration.
Elastica Models for Color Image Regularization

Hao Liu,Xue-Cheng Tai,Ron Kimmel,Roland Glowinski

DOI: https://doi.org/10.48550/arXiv.2203.09995

2022-11-25

Abstract:One classical approach to regularize color is to tream them as two dimensional surfaces embedded in a five dimensional spatial-chromatic space. In this case, a natural regularization term arises as the image surface area. Choosing the chromatic coordinates as dominating over the spatial ones, the image spatial coordinates could be thought of as a paramterization of the image surface manifold in a three dimensional color space. Minimizing the area of the image manifold leads to the Beltrami flow or mean curvature flow of the image surface in the 3D color space, while minimizing the elastica of the image surface yields an additional interesting regularization. Recently, the authors proposed a color elastica model, which minimizes both the surface area and elastica of the image manifold. In this paper, we propose to modify the color elastica and introduce two new models for color image regularization. The revised measures are motivated by the relations between the color elastica model, Euler's elastica model and the total variation model for gray level images. Compared to our previous color elastica model, the new models are direct extensions of Euler's elastica model to color images. The proposed models are nonlinear and challenging to minimize. To overcome this difficulty, two operator-splitting methods are suggested. Specifically, nonlinearities are decoupled by introducing new vector- and matrix-valued variables. Then, the minimization problems are converted to solving initial value problems which are time-discretized by operator splitting. Each subproblem, after splitting either, has a closed-form solution or can be solved efficiently. The effectiveness and advantages of the proposed models are demonstrated by comprehensive experiments. The benefits of incorporating the elastica of the image surface as regularization terms compared to common alternatives are empirically validated.

Computer Vision and Pattern Recognition,Numerical Analysis
Better than the Total Variation Regularization

Gengsheng L Zeng

Abstract:The total variation (TV) regularization is popular in iterative image reconstruction when the piecewise-constant nature of the image is encouraged. As a matter of fact, the TV regularization is not strong enough to enforce the piecewise-constant appearance. This paper suggests a different regularization function that is able to discourage some smooth transitions in the image and to encourage the piecewise-constant behavior. This new regularization function involves a Gaussian function. We use the limited-angle tomography problem to illustrate the effectiveness of this new regularization function. The limited-angle tomography situation considered in this paper uses a scanning angular range of 40 ° . For two-dimensional parallel-beam imaging, the required angular range is supposed to be 180 ° .
Modular proximal optimization for multidimensional total-variation regularization

Álvaro Barbero,Suvrit Sra

DOI: https://doi.org/10.48550/arXiv.1411.0589

2017-12-31

Abstract:We study \emph{TV regularization}, a widely used technique for eliciting structured sparsity. In particular, we propose efficient algorithms for computing prox-operators for $\ell_p$-norm TV. The most important among these is $\ell_1$-norm TV, for whose prox-operator we present a new geometric analysis which unveils a hitherto unknown connection to taut-string methods. This connection turns out to be remarkably useful as it shows how our geometry guided implementation results in efficient weighted and unweighted 1D-TV solvers, surpassing state-of-the-art methods. Our 1D-TV solvers provide the backbone for building more complex (two or higher-dimensional) TV solvers within a modular proximal optimization approach. We review the literature for an array of methods exploiting this strategy, and illustrate the benefits of our modular design through extensive suite of experiments on (i) image denoising, (ii) image deconvolution, (iii) four variants of fused-lasso, and (iv) video denoising. To underscore our claims and permit easy reproducibility, we provide all the reviewed and our new TV solvers in an easy to use multi-threaded C++, Matlab and Python library.

Machine Learning,Optimization and Control
Construction Model for Total Variation Regularization Parameter

Guanghua Gong,Hongming Zhang,Minyu Yao

DOI: https://doi.org/10.1364/oe.22.010500

IF: 3.8

2014-01-01

Optics Express

Abstract:Image denoising is important for high-quality imaging in adaptive optics. Richardson-Lucy deconvolution with total variation(TV) regularization is commonly used in image denoising. The selection of TV regularization parameter is an essential issue, yet no systematic approach has been proposed. A construction model for TV regularization parameter is proposed in this paper. It consists of four fundamental elements, the properties of which are analyzed in details. The proposed model bears generality, making it apply to different image recovery scenarios. It can achieve effective spatially adaptive image recovery, which is reflected in both noise suppression and edge preservation. Simulations are provided as validation of recovery and demonstration of convergence speed and relative mean-square error.
An Image Segmentation Model with Transformed Total Variation

Elisha Dayag,Kevin Bui,Fredrick Park,Jack Xin

2024-06-04

Abstract:Based on transformed $\ell_1$ regularization, transformed total variation (TTV) has robust image recovery that is competitive with other nonconvex total variation (TV) regularizers, such as TV$^p$, $0<p<1$. Inspired by its performance, we propose a TTV-regularized Mumford--Shah model with fuzzy membership function for image segmentation. To solve it, we design an alternating direction method of multipliers (ADMM) algorithm that utilizes the transformed $\ell_1$ proximal operator. Numerical experiments demonstrate that using TTV is more effective than classical TV and other nonconvex TV variants in image segmentation.

Computer Vision and Pattern Recognition,Image and Video Processing,Numerical Analysis
Data-proximal complementary $\ell^1$-TV reconstruction for limited data CT

Simon Göppel,Jürgen Frikel,Markus Haltmeier

2023-06-18

Abstract:In a number of tomographic applications, data cannot be fully acquired, resulting in a severely underdetermined image reconstruction. In such cases, conventional methods lead to reconstructions with significant artifacts. To overcome these artifacts, regularization methods are applied that incorporate additional information. An important example is TV reconstruction, which is known to be efficient at compensating for missing data and reducing reconstruction artifacts. At the same time, however, tomographic data is also contaminated by noise, which poses an additional challenge. The use of a single regularizer must therefore account for both the missing data and the noise. However, a particular regularizer may not be ideal for both tasks. For example, the TV regularizer is a poor choice for noise reduction across multiple scales, in which case $\ell^1$ curvelet regularization methods are well suited. To address this issue, in this paper we introduce a novel variational regularization framework that combines the advantages of different regularizers. The basic idea of our framework is to perform reconstruction in two stages, where the first stage mainly aims at accurate reconstruction in the presence of noise, and the second stage aims at artifact reduction. Both reconstruction stages are connected by a data proximity condition. The proposed method is implemented and tested for limited-view CT using a combined curvelet-TV approach. We define and implement a curvelet transform adapted to the limited-view problem and illustrate the advantages of our approach in numerical experiments.

Numerical Analysis
A nonconvex regularization model and the ADMM based algorithm

Zhuang Fang,Tang Liming,Wu Liang,Liu Hanxin

DOI: https://doi.org/10.1038/s41598-022-11938-7

IF: 4.6

2022-05-13

Scientific Reports

Abstract:The total variation (TV) regularization with l 1 fidelity is a popular method to restore the image contaminated by salt and pepper noise, but it often suffers from limited performance in edge-preserving. To solve this problem, we propose a nonconvex TV q - l 1 regularization model in this paper, which utilizes a nonconvex l q -norm $$(0<q<1)$$ ( 0 < q < 1 ) defined in total variation (TV) domain (called TV q regularizer) to regularize the restoration, and uses l 1 fidelity to measure the noise. Compared to the traditional TV model, the proposed model can more effectively preserve edges and contours since it provides a more sparse representation of the restoration in TV domain. An alternating direction method of multipliers (ADMM) combining with majorization-minimization (MM) scheme and proximity operator is introduced to numerically solve the proposed model. In particular, a sufficient condition for the convergence of the proposed algorithm is provided. Numerical results validate the proposed model and algorithm, which can effectively remove salt and pepper noise while preserving image edges and contours. In addition, compared with several state-of-the-art variational regularization models, the proposed model shows the best performance in terms of peak signal to noise ratio (PSNR) and mean structural similarity index (MSSIM). We can obtain about 0.5 dB PSNR and 0.06 MSSIM improvements against all compared models.</q<1)$$

multidisciplinary sciences
Learning Weakly Convex Regularizers for Convergent Image-Reconstruction Algorithms

Alexis Goujon,Sebastian Neumayer,Michael Unser

DOI: https://doi.org/10.1137/23m1565243

IF: 1.938

2024-01-19

SIAM Journal on Imaging Sciences

Abstract:SIAM Journal on Imaging Sciences, Volume 17, Issue 1, Page 91-115, March 2024. We propose to learn non-convex regularizers with a prescribed upper bound on their weak-convexity modulus. Such regularizers give rise to variational denoisers that minimize a convex energy. They rely on few parameters (less than 15,000) and offer a signal-processing interpretation as they mimic handcrafted sparsity-promoting regularizers. Through numerical experiments, we show that such denoisers outperform convex-regularization methods as well as the popular BM3D denoiser. Additionally, the learned regularizer can be deployed to solve inverse problems with iterative schemes that provably converge. For both CT and MRI reconstruction, the regularizer generalizes well and offers an excellent tradeoff between performance, number of parameters, guarantees, and interpretability when compared to other data-driven approaches.

mathematics, applied,computer science, software engineering, artificial intelligence,imaging science & photographic technology
Spectral Total-Variation Processing of Shapes: Theory and Applications

Jonathan Brokman,Martin Burger,Guy Gilboa

DOI: https://doi.org/10.1145/3641845

2024-02-02

Abstract:We present an analysis of total-variation (TV) on non-Euclidean parameterized surfaces, a natural representation of the shapes used in 3D graphics. Our work explains recent experimental findings in shape spectral TV [Fumero et al., 2020] and adaptive anisotropic spectral TV [Biton and Gilboa, 2022]. A new way to generalize set convexity from the plane to surfaces is derived by characterizing the TV eigenfunctions on surfaces. Relationships between TV, area, eigenvalue, eigenfunctions and their discontinuities are discovered. Further, we expand the shape spectral TV toolkit to include versatile zero-homogeneous flows demonstrated through smoothing and exaggerating filters. Last but not least, we propose the first TV-based method for shape deformation, characterized by deformations along geometrical bottlenecks. We show these bottlenecks to be aligned with eigenfunction discontinuities. This research advances the field of spectral TV on surfaces and its application in 3D graphics, offering new perspectives for shape filtering and deformation.

Computational Geometry
Anisotropic Huber-L1 Optical Flow

H. Bischof,Werner Trobin,D. Cremers,T. Pock,Manuel Werlberger,A. Wedel

DOI: https://doi.org/10.5244/C.23.108

Abstract:TV regularization is an L1 penalization of the flow gradient magnitudes, and due to the tendency of the L1 norm to favor sparse solutions (i.e. lots of ‘zeros’), the fill-in effect caused by the regularizer leads to piecewise constant solutions in weakly textured areas. This effect, known as ‘staircasing’ in a 1D setting, can be reduced significantly by using a quadratic penalization for small gradient magnitudes while sticking to linear penalization for larger magnitudes to maintain the discontinuity preserving properties known from TV. A comparison of isotropic TV and isotropic Huber regularity is shown in Fig. 1 by means of rendering the disparities u1 of the Dimetrodon dataset. The color coded flow (cf. Fig. 1(a)) is superimposed as texture. Based on the two observations that motion discontinuities often occur along object boundaries and that in turn object boundaries often coincide

Mathematics,Physics,Computer Science
Multiple Degree Total Variation (MDTV) Regularization for Image Restoration.

Yue Hu,Xin Lu,Mathews Jacob

DOI: https://doi.org/10.1109/icip.2016.7532700

2016-01-01

Abstract:We introduce a novel image regularization termed as multiple degree total variation (MDTV). This type of regularization combines the first and second degree directional derivatives, thus providing a good balance between preservation of edges and region smoothness. In order to solve the resulting optimization problem, we propose an iteratively reweighted majorize minimize algorithm. We demonstrate the utility of the MDTV regularization in the context of image denoising and compressed sensing image reconstruction. We compare the proposed method with the standard TV, and the state of the art higher degree methods, including higher degree total variation (HDTV) and total generalized variation (TGV) based schemes. Numerical results indicate that the MDTV penalty provides improved image recovery performance.

Convex Regularization of Multi-Channel Images Based on Variants of the TV-Model

Convex and non-convex adaptive TV regularizations for color image restoration

Universal Minimality in TV Regularization with Rectilinear Anisotropy

Identifiability of Multichannel Blind Deconvolution and Nonconvex Regularization Algorithm

Regularization with Sparse Vector Fields: From Image Compression to TV-type Reconstruction

Non-convex regularization based on shrinkage penalty function

Fast Optimization for Multichannel Total Variation Minimization with Non-Quadratic Fidelity

A Tensor-Based Nonlocal Total Variation Model for Multi-Channel Image Recovery

Image Restoration by Combined Order Regularization with Optimal Spatial Adaptation

Image Restoration Via Multi-Scale Non-Local Total Variation Regularization

Elastica Models for Color Image Regularization

Better than the Total Variation Regularization

Modular proximal optimization for multidimensional total-variation regularization

Construction Model for Total Variation Regularization Parameter

An Image Segmentation Model with Transformed Total Variation

Data-proximal complementary $\ell^1$-TV reconstruction for limited data CT

A nonconvex regularization model and the ADMM based algorithm

Learning Weakly Convex Regularizers for Convergent Image-Reconstruction Algorithms

Spectral Total-Variation Processing of Shapes: Theory and Applications

Anisotropic Huber-L1 Optical Flow

Multiple Degree Total Variation (MDTV) Regularization for Image Restoration.