Solving Inverse Problems with Score-Based Generative Priors learned from Noisy Data

Asad Aali,Marius Arvinte,Sidharth Kumar,Jonathan I. Tamir

DOI: https://doi.org/10.1109/IEEECONF59524.2023.10477042

2023-05-02

Abstract:We present SURE-Score: an approach for learning score-based generative models using training samples corrupted by additive Gaussian noise. When a large training set of clean samples is available, solving inverse problems via score-based (diffusion) generative models trained on the underlying fully-sampled data distribution has recently been shown to outperform end-to-end supervised deep learning. In practice, such a large collection of training data may be prohibitively expensive to acquire in the first place. In this work, we present an approach for approximately learning a score-based generative model of the clean distribution, from noisy training data. We formulate and justify a novel loss function that leverages Stein's unbiased risk estimate to jointly denoise the data and learn the score function via denoising score matching, while using only the noisy samples. We demonstrate the generality of SURE-Score by learning priors and applying posterior sampling to ill-posed inverse problems in two practical applications from different domains: compressive wireless multiple-input multiple-output channel estimation and accelerated 2D multi-coil magnetic resonance imaging reconstruction, where we demonstrate competitive reconstruction performance when learning at signal-to-noise ratio values of 0 and 10 dB, respectively.

Machine Learning,Image and Video Processing,Signal Processing

What problem does this paper attempt to address?

The core problem that this paper attempts to solve is: how to use training samples only polluted by Gaussian noise to learn a score function that can effectively solve linear inverse problems. Specifically, the paper proposes a method named SURE - Score. This method can learn the score generation model of clean data distribution through joint denoising and score matching in the case of only noisy data. This is very important in practical applications because in many cases, obtaining a large amount of high - quality training data without noise is very difficult or costly.

The main contributions of the paper include:
1. **SURE - Score Framework**: A new loss function is proposed, which combines Stein's Unbiased Risk Estimate (SURE) and Denoising Score Matching (DSM) to simultaneously achieve denoising and score function learning in a deep neural network.
2. **Experimental Verification**: The effectiveness of the SURE - Score method is verified in two different application areas - compressed multi - input multi - output (MIMO) channel estimation and accelerated two - dimensional multi - coil magnetic resonance imaging (MRI) reconstruction. Even when the signal - to - noise ratio (SNR) is 0 dB, SURE - Score can learn accurate priors and be used to solve linear inverse problems.
3. **Performance Evaluation**: By comparing with baseline methods such as supervised learning, simple DSM training, BM3D, Noise2Self and Noise2Score, the superior performance of SURE - Score in denoising and posterior reconstruction is demonstrated.

In conclusion, this paper provides an effective solution for learning high - quality score generation models when only noisy data is available, which is of great significance for many practical application scenarios.