SeGFusion: A semantic saliency guided infrared and visible image fusion method
Jinxin Xiong,Gang Liu,Haojie Tang,Xinjie Gu,Durga Prasad Bavirisetti
DOI: https://doi.org/10.1016/j.infrared.2024.105344
IF: 2.997
2024-05-25
Infrared Physics & Technology
Abstract:When infrared targets are located at the edge of an image or when the targets are relatively small, the standard infrared and visible image fusion algorithm becomes a major problem because it relies on manually designed strategies and low-level image statistics for saliency detection. To address this issue, SeGFuison is proposed. It is a semantic saliency guided infrared and visible image fusion method composed of an autoencoder, a fusion layer, and a Semantic Segmentation-based Deep Saliency model (SSDS). It focuses on the structural information of images and generates saliency maps at the feature level, so that infrared targets can be extracted more accurately, thereby avoiding the introduction of artifacts and noise in fusion images. Incorporating saliency maps dynamically generated by SSDS, our approach effectively guides the training process of the fusion model. This strategic utilization guarantees that the resulting fused image maintains a saliency map that closely resembles that of the original infrared image. Furthermore, saliency maps are employed to partition images into distinct regions, namely target areas and background areas. This segmentation enables the design of distinct loss functions tailored to the unique characteristics of each area. As a result, our approach ensures the fusion of images preserves both salient targets and intricate background details, thus upholding a comprehensive depiction of fusion information. Through rigorous experimentation conducted on widely recognized public datasets including TNO, RoadScene, and MSRS, our algorithm has exhibited distinct advantages over contemporary state-of-the-art algorithms, both in terms of objective metrics and subjective evaluations. Notably, SeGFusion attains remarkable scores on key indicators such as FMI, VIF, and SD, affirming its superiority. Furthermore, it excels in subjective assessments, producing fused images of unparalleled clarity. The obtained experimental results compellingly showcase the inherent potential of our proposed algorithm, thereby substantiating its viability for diverse applications within fields such as infrared instruments and equipment.
optics,physics, applied,instruments & instrumentation