VIFNet: An End-to-end Visible-Infrared Fusion Network for Image Dehazing

Meng Yu,Te Cui,Haoyang Lu,Yufeng Yue

2024-04-11

Abstract:Image dehazing poses significant challenges in environmental perception. Recent research mainly focus on deep learning-based methods with single modality, while they may result in severe information loss especially in dense-haze scenarios. The infrared image exhibits robustness to the haze, however, existing methods have primarily treated the infrared modality as auxiliary information, failing to fully explore its rich information in dehazing. To address this challenge, the key insight of this study is to design a visible-infrared fusion network for image dehazing. In particular, we propose a multi-scale Deep Structure Feature Extraction (DSFE) module, which incorporates the Channel-Pixel Attention Block (CPAB) to restore more spatial and marginal information within the deep structural features. Additionally, we introduce an inconsistency weighted fusion strategy to merge the two modalities by leveraging the more reliable information. To validate this, we construct a visible-infrared multimodal dataset called AirSim-VID based on the AirSim simulation platform. Extensive experiments performed on challenging real and simulated image datasets demonstrate that VIFNet can outperform many state-of-the-art competing methods. The code and dataset are available at

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The main problem this paper attempts to address is the issue of information loss in image dehazing, especially under high-density haze conditions. Existing single-modal deep learning methods may lead to severe information loss when dealing with high-density haze, affecting the quality of image restoration. Infrared images have strong robustness against haze, but existing methods mainly use the infrared modality as auxiliary information, failing to fully utilize its rich information for dehazing.

To tackle this challenge, the paper proposes a Visible-Infrared Fusion Network (VIFNet) for image dehazing. Specifically, the main contributions of the paper include:

1. **Proposing an end-to-end multimodal fusion dehazing framework** aimed at restoring high-quality images. Additionally, a visible-infrared dataset (AirSim-VID) containing 3 different types of haze concentrations is provided based on the AirSim simulation platform.

2. **Introducing a multi-scale deep structure feature extraction (DSFE) module** in the deep feature extraction stage, which combines a Channel-Pixel Attention Block (CPAB) to explore more spatial and edge information in the feature maps.

3. **Introducing an efficient inconsistency fusion strategy** in the feature weighting fusion stage, which emphasizes more reliable and consistent information by adjusting the fusion weights between the two modalities.

Through these innovations, VIFNet can effectively restore image quality under high-density haze conditions and demonstrates superior performance over existing methods on multiple datasets.