Abstract:Existing learning-based denoising methods typically train models to generalize the image prior from large-scale datasets, suffering from the variability in noise distributions encountered in real-world scenarios. In this work, we propose a new perspective on the denoising challenge by highlighting the distinct separation between noise and image priors. This insight forms the basis for our development of conditional optimization framework, designed to overcome the constraints of traditional denoising framework. To this end, we introduce a Locally Noise Prior Estimation (LoNPE) algorithm, which accurately estimates the noise prior directly from a single raw noisy image. This estimation acts as an explicit prior representation of the camera sensor's imaging environment, distinct from the image prior of scenes. Additionally, we design an auxiliary learnable LoNPE network tailored for practical application to sRGB noisy images. Leveraging the estimated noise prior, we present a novel Conditional Denoising Transformer (Condformer), by incorporating the noise prior into a conditional self-attention mechanism. This integration allows the Condformer to segment the optimization process into multiple explicit subspaces, significantly enhancing the model's generalization and flexibility. Extensive experimental evaluations on both synthetic and real-world datasets, demonstrate that the proposed method achieves superior performance over current state-of-the-art methods. The source code is available at <a class="link-external link-https" href="https://github.com/YuanfeiHuang/Condformer" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

### Problems Addressed by the Paper This paper aims to address the issue of noise distribution variation encountered by existing learning-based image denoising methods in real-world scenarios. Specifically, current denoising methods typically train models by generalizing image priors from large-scale datasets, but in the real world, variations in noise distribution limit the effectiveness of these methods. The paper proposes a new perspective by distinguishing between noise priors and image priors and designs a conditional optimization framework to overcome the limitations of traditional denoising frameworks. ### Main Contributions 1. **Rethinking Imaging Mechanism**: Rethinking the imaging mechanism from a physical perspective, emphasizing the independence of noise priors and image priors. This distinction is particularly important for conditional optimization, especially in practical application scenarios. 2. **Introducing the LoNPE Algorithm**: Proposing a Local Noise Prior Estimation (LoNPE) algorithm to estimate noise priors from a single raw noisy image. This method effectively captures the characteristics of sensor noise, providing explicit prior representation for conditional optimization. Additionally, a learnable LoNPE network is designed for practical applications with only one sRGB noisy image. 3. **Designing the Condformer Model**: Proposing a Conditional Denoising Transformer (Condformer) that embeds noise priors in the conditional self-attention module, partitioning the entire optimization space into multiple explicit optimization subspaces. This design significantly enhances the model's generalization ability and flexibility. 4. **Experimental Validation**: Demonstrating the superior performance of the LoNPE algorithm and Condformer model in various real and synthetic noise analysis and image denoising tasks through quantitative and qualitative experiments. ### Method Overview 1. **Preliminary Introduction to Noise Priors**: - **Noise Formation Model**: Detailed introduction to the noise formation model in camera sensors, including shot noise during the photon-to-electron conversion process and readout noise during the electron-to-digital conversion process. - **Independence of Noise Priors and Image Priors**: Experimental validation showing that noise priors mainly depend on imaging conditions (such as ISO levels and shutter speed) and are independent of image priors (such as scene brightness). 2. **Local Noise Prior Estimation (LoNPE)**: - **Algorithm Workflow**: Estimating noise priors from a single noisy image through steps like preprocessing, local smooth block selection, and statistical value calculation. - **Network Design**: Constructing a learnable CNN model (LoNPE network) to predict noise priors from a single sRGB noisy image. 3. **Conditional Denoising Transformer (Condformer)**: - **Embedding Noise Priors**: Embedding noise priors in the latent space and optimizing through the conditional self-attention module. - **Overall Workflow**: Using Restormer as the baseline model, replacing its self-attention module with a conditional self-attention module, and incorporating noise priors for denoising. Employing a hierarchical skip connection strategy to retain fine structures and texture details in the restored image. ### Conclusion By introducing noise priors and a conditional optimization framework, the paper significantly improves image denoising performance. Experimental results show that the LoNPE algorithm and Condformer model perform excellently in various noise analysis and image denoising tasks. Future work will further explore noise prior modeling and optimization methods to enhance the robustness and generalization ability of the model.

Beyond Image Prior: Embedding Noise Prior into Conditional Denoising Transformer

CFNet: Conditional Filter Learning with Dynamic Noise Estimation for Real Image Denoising

Boosting Noise Reduction Effect Via Unsupervised Fine-Tuning Strategy

An Adaptive Image Denoising Method Based on Deep Rectified Denoising Auto-Encoder

External Prior Guided Internal Prior Learning for Real-World Noisy Image Denoising

Optimizing Image Compression via Joint Learning with Denoising

Remote Sensing Image Mixed Noise Denoising with Noise Parameter Estimation

A Prior-Guided Deep Network for Real Image Denoising and Its Applications

Self-Supervised Image Denoising for Real-World Images with Context-aware Transformer

Estimating Fine-Grained Noise Model via Contrastive Learning

Enhancing and Learning Denoiser Without Clean Reference

LAN: Learning to Adapt Noise for Image Denoising

Image Denoising Using a Novel Deep Generative Network with Multiple Target Images and Adaptive Termination Condition

A Triple Deep Image Prior Model for Image Denoising Based on Mixed Priors and Noise Learning

Neighbor2Neighbor: A Self-Supervised Framework for Deep Image Denoising

An efficient lightweight network for image denoising using progressive residual and convolutional attention feature fusion

Hierarchical Disentangled Representation for Image Denoising and Beyond

Boosting Image Denoising Effect Via Low-Level Noise Injection

Noisy-as-Clean: Learning Self-Supervised Denoising From Corrupted Image

Image-Specific Prior Adaptation for Denoising

Positive2Negative: Breaking the Information-Lossy Barrier in Self-Supervised Single Image Denoising