Abstract:In this work, we observe that model trained on vast general images via masking strategy, has been naturally embedded with their distribution knowledge, thus spontaneously attains the underlying potential for strong image denoising. Based on this observation, we propose a novel zero-shot denoising paradigm, i.e., Masked Pre-train then Iterative fill (MPI). MPI first trains model via masking and then employs pre-trained weight for high-quality zero-shot image denoising on a single noisy image. Concretely, MPI comprises two key procedures: 1) Masked Pre-training involves training model to reconstruct massive natural images with random masking for generalizable representations, gathering the potential for valid zero-shot denoising on images with varying noise degradation and even in distinct image types. 2) Iterative filling exploits pre-trained knowledge for effective zero-shot denoising. It iteratively optimizes the image by leveraging pre-trained weights, focusing on alternate reconstruction of different image parts, and gradually assembles fully denoised image within limited number of iterations. Comprehensive experiments across various noisy scenarios underscore the notable advances of MPI over previous approaches with a marked reduction in inference time. Code available at <a class="link-external link-https" href="https://github.com/krennic999/MPI" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve several key problems in **Zero - shot Image Denoising**: 1. **The problem of relying on additional noise datasets**: - Current supervised learning methods require a large number of noisy and clean image pairs, and unsupervised methods also need to collect additional noise datasets. This not only increases the cost of data collection but may also cause the model to rely on specific noise patterns or intensities, thus limiting its performance in unknown noise situations. 2. **Limitations of existing zero - shot methods**: - Existing zero - shot denoising methods usually need to train new networks from scratch for each noisy image, which brings two main problems: - **Insufficient information**: A single image provides limited information, resulting in denoising results that may be too blurry, have noise artifacts, or be of poor quality. - **Time - consuming**: Retraining the network for each image is very time - consuming, usually taking several minutes or even longer. 3. **Improving denoising quality and inference speed**: - The paper proposes a new zero - shot denoising paradigm, namely **Masked Pre - train then Iterative fill (MPI)**, which improves denoising quality and inference speed by introducing Masked Pre - training, especially when dealing with unseen noise types. ### Specific solutions To overcome the above problems, the paper proposes the following innovations: - **Masked Pre - training**: - Use large - scale natural images for masked pre - training, enabling the model to learn the distribution knowledge of natural images. This pre - training method makes the model have potential denoising capabilities without the need to retrain for each noisy image. - **Iterative Filling**: - In the zero - shot inference stage, use the pre - trained weights to optimize a single noisy image. By iteratively optimizing the reconstruction of different image regions, a fully denoised image is gradually assembled. This method not only improves the denoising quality but also significantly reduces the inference time. - **Broad generalization ability**: - Experiments show that MPI performs well not only on synthetic noise but also can effectively handle noise in the real world (such as in the SIDD benchmark) and in medical images, demonstrating its strong generalization ability. ### Main contributions 1. **Propose a new zero - shot denoising paradigm MPI**, which applies masked pre - training to zero - shot denoising tasks for the first time, while improving image quality and inference speed. 2. **Develop a pre - training scheme based on pixel - level random masks**, and perform zero - shot inference on specific noisy images through iterative filling, efficiently generating high - quality denoising results. 3. **Extensive experimental verification**, proving that MPI has superior performance, efficiency, and robustness under multiple noise types, and is suitable for practical applications. Through these innovations, the paper provides a more general and efficient zero - shot denoising solution, solving some key problems in existing methods.

Masked Pre-training Enables Universal Zero-shot Denoiser

Masked Image Training for Generalizable Deep Image Denoising

Multinoise-type Blind Denoising Using a Single Uniform Deep Convolutional Neural Network.

Learning Mask-aware CLIP Representations for Zero-Shot Segmentation

Learning a Model-Based Deep Hyperspectral Denoiser from a Single Noisy Hyperspectral Image.

MM-BSN: Self-Supervised Image Denoising for Real-World with Multi-Mask based on Blind-Spot Network

Revealing the Denoising Principle of Zero-Shot N2N-Based Algorithm from 1D Spectrum to 2D Image

Asymmetric Mask Scheme for Self-Supervised Real Image Denoising

Real-time Monte Carlo Denoising with Weight Sharing Kernel Prediction Network

One-Pot Multi-frame Denoising

Zero-Shot Noise2Noise: Efficient Image Denoising without any Data

Plug-and-Play Image Restoration with Deep Denoiser Prior

Masked Momentum Contrastive Learning for Zero-shot Semantic Understanding

A Two-Stage Progressive Pre-training using Multi-Modal Contrastive Masked Autoencoders

View Blind-spot as Inpainting: Self-Supervised Denoising with Mask Guided Residual Convolution

Few-Shot Learning for Image Denoising

Joint Demosaicking and Denoising Benefits from a Two-stage Training Strategy

Learning Multi‐Scale Deep Image Prior for High‐Quality Unsupervised Image Denoising

Physics-guided Noise Neural Proxy for Practical Low-light Raw Image Denoising

Self-supervised deep learning for joint 3D low-dose PET/CT image denoising

Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising