Abstract:This work addresses the fundamental linear inverse problem in compressive sensing (CS) by introducing a new type of regularizing generative prior. Our proposed method utilizes ideas from classical dictionary-based CS and, in particular, sparse Bayesian learning (SBL), to integrate a strong regularization towards sparse solutions. At the same time, by leveraging the notion of conditional Gaussianity, it also incorporates the adaptability from generative models to training data. However, unlike most state-of-the-art generative models, it is able to learn from a few compressed and noisy data samples and requires no optimization algorithm for solving the inverse problem. Additionally, similar to Dirichlet prior networks, our model parameterizes a conjugate prior enabling its application for uncertainty quantification. We support our approach theoretically through the concept of variational inference and validate it empirically using different types of compressible signals.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the basic linear inverse problem in Compressive Sensing (CS). Specifically, the author proposes a new regularized generative prior model, aiming to learn from a small number of compressed and noisy data samples and use it to solve the linear inverse problem. The following is a detailed interpretation of the core problems and solutions in the paper:
### Problems the paper attempts to solve
1. **Solving the linear inverse problem**:
- In compressive sensing, the goal is to recover the signal \(x^*\in\mathbb{R}^N\) from under - determined measurements \(y\in\mathbb{R}^M\), where \(M\ll N\).
- The measurement relationship is \(y = Ax^*+n\), where \(A\) is the measurement matrix and \(n\sim\mathcal{N}(0,\sigma_n^2I)\) is the additive Gaussian white noise.
2. **Learning from a small number of compressed and noisy data**:
- In many application scenarios, it may be impossible or too costly to obtain a large amount of ground - truth data, such as in electron microscopes, wearable electrocardiogram devices, etc.
- Therefore, a method is needed that can learn effective prior knowledge from a small number of compressed and noisy data samples to solve the inverse problem.
3. **Combining sparsity and generative models**:
- Existing CS algorithms such as Lasso regression and Orthogonal Matching Pursuit (OMP) rely on the sparsity assumption of the signal.
- Modern deep - learning methods such as unfolding algorithms, CS based on generative models, and untrained Neural Networks (NNs) expand the possibilities by learning prior knowledge from the training set.
- The method proposed in this paper combines the advantages of Sparse Bayesian Learning (SBL) and generative models, being able to learn sparse representations and having the adaptability of generative models.
### Proposed solutions
1. **A new sparse generative prior model**:
- The author introduces a new generative prior model that can learn from a small number of compressed and noisy data samples and does not require an optimization algorithm to solve the inverse problem.
- The model utilizes conditional Gaussianity and variational inference and achieves uncertainty quantification through parameterized conjugate prior.
2. **Theoretical support and verification**:
- The author proves through variational inference that the training of the model maximizes a variational lower bound of the sparse - induced log - evidence.
- Two specific implementation methods are proposed: sparse Bayesian generative models based on Variational Auto - Encoders (VAEs) and Gaussian Mixture Models (GMMs).
- Experiments verify the performance of this method on different types of compressible signals.
### Formula presentation
- **Measurement model**:
\[
y = Ax^*+n
\]
where \(y\in\mathbb{R}^M\), \(A\in\mathbb{R}^{M\times N}\), \(n\sim\mathcal{N}(0,\sigma_n^2I)\).
- **Prior and posterior in Sparse Bayesian Learning**:
\[
p(y|s^*)=\mathcal{N}(y; ADs^*, \sigma^2I),\quad s^*\sim\mathcal{N}(s; 0,\text{diag}(\gamma))
\]
- **Variational lower bound (ELBO)**:
\[
\log p_\theta(y_i)\geq L_{\text{CSVAE}}(\theta,\phi)=\mathbb{E}_{q_\phi(z,s|y_i)}\left[\log\frac{p(y_i|s)p_\theta(s|z)