Abstract:When infrared targets are located at the edge of an image or when the targets are relatively small, the standard infrared and visible image fusion algorithm becomes a major problem because it relies on manually designed strategies and low-level image statistics for saliency detection. To address this issue, SeGFuison is proposed. It is a semantic saliency guided infrared and visible image fusion method composed of an autoencoder, a fusion layer, and a Semantic Segmentation-based Deep Saliency model (SSDS). It focuses on the structural information of images and generates saliency maps at the feature level, so that infrared targets can be extracted more accurately, thereby avoiding the introduction of artifacts and noise in fusion images. Incorporating saliency maps dynamically generated by SSDS, our approach effectively guides the training process of the fusion model. This strategic utilization guarantees that the resulting fused image maintains a saliency map that closely resembles that of the original infrared image. Furthermore, saliency maps are employed to partition images into distinct regions, namely target areas and background areas. This segmentation enables the design of distinct loss functions tailored to the unique characteristics of each area. As a result, our approach ensures the fusion of images preserves both salient targets and intricate background details, thus upholding a comprehensive depiction of fusion information. Through rigorous experimentation conducted on widely recognized public datasets including TNO, RoadScene, and MSRS, our algorithm has exhibited distinct advantages over contemporary state-of-the-art algorithms, both in terms of objective metrics and subjective evaluations. Notably, SeGFusion attains remarkable scores on key indicators such as FMI, VIF, and SD, affirming its superiority. Furthermore, it excels in subjective assessments, producing fused images of unparalleled clarity. The obtained experimental results compellingly showcase the inherent potential of our proposed algorithm, thereby substantiating its viability for diverse applications within fields such as infrared instruments and equipment.

Saliency-Based Geometry Measurement for Image Fusion Performance.

Fusion of Visible and Infrared Images Using Saliency Analysis and Detail Preserving Based Image Decomposition

Multi-Frame Image Fusion Method Combining Spatial-Temporal Saliency Detection and Nsct

Salient Object Detection with High-Level Prior Based on Bayesian Fusion.

Dual Band Image Fusion Method Based on Region Saliency

Image Fusion Quality Metrics By Directional Projection

Similarity-Based Objective Measure for Performance of Image Fusion

Multi Information Fusion Network for Saliency Quality Assessment.

Salience Preserving Image Fusion with Dynamic Range Compression.

Selection of image fusion quality measures: objective, subjective, and metric assessment

A new image fusion performance metric based on visual information fidelity

Salience Preserving Multi-Focus Image Fusion

SeGFusion: A semantic saliency guided infrared and visible image fusion method

Video fusion performance evaluation based on structural similarity and human visual perception

Objective Image Fusion Quality Evaluation Using Structural Similarity

Received:2007-01-31;Revised:2007-08-30 Multi-focus image fusion with salience preserving

A Novel Remote-Sensing Image Fusion Method Based on Hybrid Visual Saliency Analysis

OPTICAL AND SAR IMAGE FUSION BASED ON VISUAL SALIENCY FEATURES

Image Quality Assessment by Decision Fusion

Shared Chain Mutual Information Measure for Image Fusion

Novel Full-Reference Image Quality Assessment Metric Based on Entropy Fusion