Infrared and visible image fusion based on saliency detection and deep multi-scale orientational features
Gang Liu,Menghan Jia,Xiao Wang,Durga Prasad Bavirisetti
DOI: https://doi.org/10.1007/s11760-024-03631-1
IF: 1.583
2024-12-04
Signal Image and Video Processing
Abstract:To effectively preserve thermal targets in infrared (IR) images and texture information in visible (VIS) images, this paper proposes a saliency detection-based and multi-scale oriented features image fusion method. Firstly, an edge guidance network is used to extract salient targets from IR images in order to generate salient target masks, which provide direction for fusing various types of information and increase the network's generalization capability. Secondly, a particular loss function in each region is developed in conjunction with the salient target masks to direct the network to perform feature extraction. It can extract salient target characteristics and background texture information selectively while maintaining the integrity of the salient target and background regions efficiently. Finally, an oriented fusion approach based on feature hierarchy is proposed, which minimizes information dropout by integrating deep features on distinct scales with direction. Extensive experiments are carried out in this research using available datasets such as TNO and RoadScene. The method outperforms 10 state-of-the-art methods in the most of the evaluation metrics. This demonstrates that proposed fusion method not only preserves the salient target information of IR images, but also retains more VIS image features.
engineering, electrical & electronic,imaging science & photographic technology