Abstract:In this work we study the behavior of the forward-backward (FB) algorithm when the proximity operator is replaced by a sub-iterative procedure to approximate a Gaussian denoiser, in a Plug-and-Play (PnP) fashion. In particular, we consider both analysis and synthesis Gaussian denoisers within a dictionary framework, obtained by unrolling dual-FB iterations or FB iterations, respectively. We analyze the associated minimization problems as well as the asymptotic behavior of the resulting FB-PnP iterations. In particular, we show that the synthesis Gaussian denoising problem can be viewed as a proximity operator. For each case, analysis and synthesis, we show that the FB-PnP algorithms solve the same problem whether we use only one or an infinite number of sub-iteration to solve the denoising problem at each iteration. To this aim, we show that each "one sub-iteration" strategy within the FB-PnP can be interpreted as a primal-dual algorithm when a warm-restart strategy is used. We further present similar results when using a Moreau-Yosida smoothing of the global problem, for an arbitrary number of sub-iterations. Finally, we provide numerical simulations to illustrate our theoretical results. In particular we first consider a toy compressive sensing example, as well as an image restoration problem in a deep dictionary framework.
Optimization and Control,Computer Vision and Pattern Recognition,Image and Video Processing,Signal Processing
What problem does this paper attempt to address?
### Problems the paper attempts to solve
This paper aims to study the behavior of the Forward - Backward (FB) algorithm when the approximate Gaussian denoiser is used as a sub - iterative process in the FB algorithm. Specifically, the paper explores two types of denoisers: Analysis Denoiser (AD) and Synthesis Denoiser (SD). These two denoisers are obtained by unfolding the dual - FB iteration or FB iteration in the dictionary framework.
The main problems include:
1. **Mathematical properties of denoisers**: Analyze the minimization problems related to AD and SD and their asymptotic behavior.
2. **Effectiveness of Plug - and - Play (PnP) algorithm**: Verify whether the FB - PnP algorithm can solve the same problem when using a single or an infinite number of sub - iterations to solve the denoising problem.
3. **Algorithm convergence**: Prove that when using a one - sub - iteration strategy, the FB - PnP algorithm can be interpreted as a primal - dual algorithm with a warm - restart strategy and can converge to the solution of the problem.
4. **Smoothing problem**: Study the behavior of the FB - PnP algorithm when using the global problem after Moreau - Yosida smoothing.
### Specific problem description
Linear inverse problems play a crucial role in fields such as imaging, neuroscience, and astrophysics. Given a degraded linear system:
\[y = Ax+\varepsilon w,\]
where \(y\in\mathbb{R}^M\) represents the degraded observation, \(A:\mathbb{R}^N\rightarrow\mathbb{R}^M\) is the linear measurement operator, \(w\in\mathbb{R}^M\) is an independently and identically distributed standard normal random variable, and \(\varepsilon > 0\). The goal is to estimate the unknown signal \(x\in\mathbb{R}^N\) from the degraded observation \(y\).
To deal with problems such as undersampling and noise, prior knowledge is usually used to guide the selection of solutions. A widely adopted method is the Maximum A Posteriori (MAP) strategy, which defines \(\hat{x}\) as the solution to the following optimization problem:
\[ \text{find}\; \hat{x}\in\arg\min_{x\in\mathbb{R}^N}\frac{1}{2}\|Ax - y\|^2 + p(x),\]
where \(p:\mathbb{R}^N\rightarrow(-\infty,+\infty]\) is a convex, lower - semicontinuous, proper function representing prior information. The choice of prior is crucial for reconstruction performance and computational complexity.
This paper focuses particularly on sparse priors, that is, the sparsity of the unknown signal \(x\) in a certain basis. The form of the prior \(p\) is:
\[p(x)=\lambda g(\Gamma x),\]
where \(\lambda > 0\) is a regularization parameter, \(\Gamma:\mathbb{R}^N\rightarrow\mathbb{R}^S\) is a linear operator, and \(g:\mathbb{R}^S\rightarrow\mathbb{R}^N\) is a convex, lower - semicontinuous, proper function used to promote sparsity in the transform domain (for example, the \(\ell_1\) norm). Common sparsifying operators include the Fourier transform, the Discrete Cosine Transform (DCT), the wavelet transform, and Total Variation (TV) regularization.
### Main contributions
1. **Properties of analysis and synthesis denoisers**: Study the minimization problems related to AD and SD under a fixed dictionary and prove that the synthesis denoising problem can be regarded as a proximity operator.
2. **Behavior of the FB - PnP algorithm**: Analyze the behavior of the FB - PnP algorithm when AD and SD are calculated by FB iteration and prove its convergence.
3. **Treatment of the smoothing problem**