Abstract:In recent years, image denoising has benefited a lot from deep neural networks. However, these models need large amounts of noisy-clean image pairs for supervision. Although there have been attempts in training denoising networks with only noisy images, existing self-supervised algorithms suffer from inefficient network training, heavy computational burden, or dependence on noise modeling. In this paper, we proposed a self-supervised framework named Neighbor2Neighbor for deep image denoising. We develop a theoretical motivation and prove that by designing specific samplers for training image pairs generation from only noisy images, we can train a self-supervised denoising network similar to the network trained with clean images supervision. Besides, we propose a regularizer in the perspective of optimization to narrow the optimization gap between the self-supervised denoiser and the supervised denoiser. We present a very simple yet effective self-supervised training scheme based on the theoretical understandings: training image pairs are generated by random neighbor sub-samplers, and denoising networks are trained with a regularized loss. Moreover, we propose a training strategy named BayerEnsemble to adapt the Neighbor2Neighbor framework in raw image denoising. The proposed Neighbor2Neighbor framework can enjoy the progress of state-of-the-art supervised denoising networks in network architecture design. It also avoids heavy dependence on the assumption of the noise distribution. We evaluate the Neighbor2Neighbor framework through extensive experiments, including synthetic experiments with different noise distributions and real-world experiments under various scenarios. The code is available online: https://github.com/TaoHuang2018/Neighbor2Neighbor.

Self-Supervised Fast Adaptation for Denoising via Meta-Learning

Spatial-Adaptive Network for Single Image Denoising

Fast Adaptation to Super-Resolution Networks via Meta-Learning

Self2Self+: Single-Image Denoising with Self-Supervised Learning and Image Quality Assessment Loss

LAN: Learning to Adapt Noise for Image Denoising

An Adaptive Image Denoising Method Based on Deep Rectified Denoising Auto-Encoder

Noise2Kernel: Adaptive Self-Supervised Blind Denoising using a Dilated Convolutional Kernel Architecture

Self-Supervised Denoising of single OCT image with Self2Self-OCT Network

Self-supervised Image Denoising with Downsampled Invariance Loss and Conditional Blind-Spot Network

Self-Supervised Learning Hyperspectral Image Denoiser with Separated Spectral-Spatial Feature Extraction.

Adaptive Image Denoising Based on Improved Stacked Sparse Denoising Auto-Encoder

Unsupervised Image Denoising with Frequency Domain Knowledge

Neighbor2Neighbor: A Self-Supervised Framework for Deep Image Denoising

Unsupervised Image Denoising in Real-World Scenarios via Self-Collaboration Parallel Generative Adversarial Branches

Toward Interactive Self-Supervised Denoising

Spatially Adaptive Self-Supervised Learning for Real-World Image Denoising

Iterative Denoiser and Noise Estimator for Self-Supervised Image Denoising

Towards Interactive Self-Supervised Denoising

Self-supervised learning for denoising of multidimensional MRI data

DETA: Denoised Task Adaptation for Few-Shot Learning

Self2self With Dropout: Learning Self-Supervised Denoising From Single Image