Abstract:Modeling and synthesizing real sRGB noise is crucial for various low-level vision tasks, such as building datasets for training image denoising systems. The distribution of real sRGB noise is highly complex and affected by a multitude of factors, making its accurate modeling extremely challenging. Therefore, recent studies have proposed methods that employ data-driven generative models, such as Generative Adversarial Networks (GAN) and Normalizing Flows. These studies achieve more accurate modeling of sRGB noise compared to traditional noise modeling methods. However, there are performance limitations due to the inherent characteristics of each generative model. To address this issue, we propose NM-FlowGAN, a hybrid approach that exploits the strengths of both GAN and Normalizing Flows. We combine pixel-wise noise modeling networks based on Normalizing Flows and spatial correlation modeling networks based on GAN. Specifically, the pixel-wise noise modeling network leverages the high training stability of Normalizing Flows to capture noise characteristics that are affected by a multitude of factors, and the spatial correlation networks efficiently model pixel-to-pixel relationships. In particular, unlike recent methods that rely on paired noisy images, our method synthesizes noise using clean images and factors that affect noise characteristics, such as easily obtainable parameters like camera type and ISO settings, making it applicable to various fields where obtaining noisy-clean image pairs is not feasible. In our experiments, our NM-FlowGAN outperforms other baselines in the sRGB noise synthesis task. Moreover, the denoising neural network trained with synthesized image pairs from our model shows superior performance compared to other baselines. Our code is available at: \url{<a class="link-external link-https" href="https://github.com/YoungJooHan/NM-FlowGAN" rel="external noopener nofollow">this https URL</a>}.

Generate Optical Flow with Conditional Generative Adversarial Network

Attack Deterministic Conditional Image Generative Models for Diverse and Controllable Generation

Conditional Inpainting Generative Flow.

SpatialGAN: Progressive Image Generation Based on Spatial Recursive Adversarial Expansion

F³A-GAN: Facial Flow for Face Animation With Generative Adversarial Networks

Flow-GAN: Combining Maximum Likelihood and Adversarial Learning in Generative Models

Dual Motion GAN for Future-Flow Embedded Video Prediction

C-Flow: Conditional Generative Flow Models for Images and 3D Point Clouds

AgeFlow: Conditional Age Progression and Regression with Normalizing Flows

Incremental Focal Loss GANs.

PCSAGAN: a physics-constrained generative network based on self-attention for high-fidelity flow field reconstruction

S-Flow GAN

Deep Learning Convective Flow Using Conditional Generative Adversarial Networks

Conditional Image-to-Video Generation with Latent Flow Diffusion Models

FlowNet: Learning Optical Flow with Convolutional Networks

Hierarchical Video Generation From Orthogonal Information: Optical Flow and Texture

FlowNAS: Neural Architecture Search for Optical Flow Estimation

Flow Matching in Latent Space

Generalization capabilities of conditional GAN for turbulent flow under changes of geometry

NM-FlowGAN: Modeling sRGB Noise without Paired Images using a Hybrid Approach of Normalizing Flows and GAN