Abstract:Smoke removal has been a challenging problem which refers to the process of retrieving clear images from the smoky ones. It has an extensive demand while the performances of existing image desmoking methods have limitations owing to the homogeneous medium based model, hand-crafted features, and insufficient datasets. Unlike the existing desmoking methods using an atmospheric scattering model, the proposed method utilizes a general deblurring model. Inspired by Generative Adversarial Network (GAN), we propose an end-to-end attentive DesmokeGAN which implements the visual attention into the generative network to effectively learn the smoke features and their surroundings. The architecture of critic network is identical to PatchGAN by adding multi-component loss function to the image patch. Due to the scarce of various and quality smoke datasets, a graphics rendering engine is used to synthesize the smoky images. The quantitative and qualitative results show that the designed framework performs better than the recent state-of-the-art desmoking approaches on both synthetic and real images in indoor and outdoor scenes, especially for the thick smoky images.

What problem does this paper attempt to address?

The paper primarily addresses the problem of single image desmoking, which is the process of recovering a clear image from one obscured by smoke. The research focuses on overcoming the limitations of existing methods, including those based on uniform medium models, handcrafted feature extraction methods, and the lack of sufficient datasets. Specifically, the paper presents the following contributions: 1. **Synthetic Smoke Dataset**: To obtain sufficient training data, the authors use Blender, a graphics rendering engine, to synthesize smoke images under different densities and conditions. This method can efficiently generate a large number of synthetic smoke image pairs without the need for manual annotation. 2. **Proposed an End-to-End Attention Mechanism Desmoking Generative Adversarial Network (DesmokeGAN)**: Unlike traditional desmoking methods, the proposed DesmokeGAN combines the concept of Generative Adversarial Networks (GAN) and introduces an attention mechanism to more effectively learn smoke features and their surrounding environment. Additionally, the network adopts a PatchGAN architecture and incorporates a multi-component loss function. 3. **Blind Smoke Blur and Additive Noise Model**: Considering the blur and color distortion caused by smoke, the study describes a kernel-free blind smoke blur model and takes into account the impact of additive noise, rather than using the traditional atmospheric scattering model. This approach may provide new insights for dealing with media of non-uniform distribution and density. 4. **Network Structure and Loss Functions**: The study details the network structure of the generator and discriminator, as well as the design of the loss functions used to train the model, including adversarial loss and perceptual loss. Through experimental validation, the method demonstrates superior desmoking performance compared to the current state-of-the-art on both synthetic and real images, particularly in handling images with thick smoke. This indicates that the method can effectively remove smoke while preserving the original image's structure and color information as much as possible.

Single Image Desmoking via Attentive Generative Adversarial Network for Smoke Detection Process

A Slight Smoke Perceptual Network.

Generative Smoke Removal

Deep Smoke Segmentation

Visual Smoke Detection Based on Ensemble Deep CNNs.

Video Smoke Detection Based on Deep Saliency Network

Attention-Aware Laparoscopic Image Desmoking Network with Lightness Embedding and Hybrid Guided Embedding

MARS-GAN: Multilevel-Feature-Learning Attention-Aware Based Generative Adversarial Network for Removing Surgical Smoke

A Self-Supervised Network-Based Smoke Removal and Depth Estimation for Monocular Endoscopic Videos

An Integrated Smoke Detection Method Based on Convolutional Neural Network and Image Processing

A hybrid attention model based on first-order statistical features for smoke recognition

Joint Generative Image Deblurring Aided by Edge Attention Prior and Dynamic Kernel Selection

SMWE-GFPNNet: A High-precision and Robust Method for Forest Fire Smoke Detection

A Smoke Removal Method Based on Combined Data and Modified U-Net for Endoscopic Images

FD-GAN: Generative Adversarial Networks with Fusion-discriminator for Single Image Dehazing

Single image dehazing algorithm based on generative adversarial network

A Gated Recurrent Network With Dual Classification Assistance for Smoke Semantic Segmentation

A lightweight network for real-time smoke semantic segmentation based on dual paths

Deep domain adaptation based video smoke detection using synthetic smoke images

Unsupervised single image dehazing with generative adversarial network

Can Image Enhancement be Beneficial to Find Smoke Images in Laparoscopic Surgery?