Abstract:Image fusion aims to integrate complementary characteristics of source images into a single fused image that better serves human visual observation and machine vision perception. However, most existing image fusion algorithms primarily focus on improving the visual appeal of fused images. Although there are some semantic-driven methods that consider semantic requirements of downstream applications, none of them have demonstrated the potential of image-level fusion compared to feature-level fusion, which fulfills high-level vision tasks directly on multi-modal features rather than on a fused image. To overcome these limitations, this paper presents a practical infrared and visible image fusion network based on progressive semantic injection and scene fidelity constraints, termed PSFusion. First of all, the sparse semantic perception branch extracts sufficient semantic features, which are then progressively integrated into the fusion network using the semantic injection module to fulfill the semantic requirements of high-level vision tasks. The scene fidelity path within the scene restoration branch is devised to ensure that the fusion features contain complete information for reconstructing the source images. Additionally, the contrast mask and salient target mask are employed to construct the fusion loss to maintain impressive visual effects of fusion results. In particular, we provide quantitative and qualitative analyses to demonstrate the potential of image-level fusion compared to feature-level fusion for high-level vision tasks. With the rapid advancement of large-scale models, image-level fusion can expeditiously leverage the advantages of multi-modal data and state-of-the-art (SOTA) unimodal segmentation to achieve superior performance. Furthermore, extensive comparative experiments demonstrate the superiority of our PSFusion over SOTA image-level fusion alternatives in terms of visual appeal and high-level semantics. Even under harsh circumstances, our method offers satisfactory fusion results to serve subsequent high-level vision applications. The source code is available at https://github.com/Linfeng-Tang/PSFusion .

Pixel Level Image Fusion Methods and Fusion Performance Evaluation

Dual Band Image Fusion Method Based on Region Saliency

The Realistic Fusion of Multi-spectral Images

A pixel-level image fusion framework based on structural similarity

Multi-Exposure Image Fusion Techniques: A Comprehensive Review

The Statistical methods of Pixel-Based Image Fusion Techniques

Remote Sensing Image Fusion Based on Fuzzy Theory in Pixel Level and Assessing the Quality of Resulting Images

Information Fusion of Multi-Sensor Images

Image Fusion Based on Multi-objective Optimization

Image Fusion Algorithm at Pixel Level Based on Edge Detection

Multi-focus image fusion: A Survey of the state of the art

Image Fusion in Remote Sensing: An Overview and Meta Analysis

Non-linear and selective fusion of cross-modal images

Deep Learning for Pixel-Level Image Fusion: Recent Advances and Future Prospects

From Multi-Scale Decomposition to Non-Multi-Scale Decomposition Methods: A Comprehensive Survey of Image Fusion Techniques and Its Applications

A new method for multi-source remote sensing image fusion

Performance of Medical Image Fusion in High-level Analysis Tasks: A Mutual Enhancement Framework for Unaligned PAT and MRI Image Fusion

Superpixel-Based Quality Assessment of Multi-Exposure Image Fusion for Both Static and Dynamic Scenes

Comparison of Pixel-Based Fusion Between Multi-Spectral Data and High Spatial Resolution Data

Rethinking the necessity of image fusion in high-level vision tasks: A practical infrared and visible image fusion network based on progressive semantic injection and scene fidelity