Abstract:The aim of infrared and visible image fusion is to generate a fused image that not only contains salient targets and rich texture details, but also facilitates high-level vision tasks. However, due to the hardware limitations of digital cameras and other devices, there are more low-resolution images in the existing datasets, and low-resolution images are often accompanied by the problem of losing details and structural information. At the same time, existing fusion algorithms focus too much on the visual quality of the fused images, while ignoring the requirements of high-level vision tasks. To address the above challenges, in this paper, we skillfully unite the super-resolution network, fusion network and segmentation network, and propose a super-resolution-based semantic-aware fusion network. First, we design a super-resolution network based on a multi-branch hybrid attention module (MHAM), which aims to enhance the quality and details of the source image, enabling the fusion network to integrate the features of the source image more accurately. Then, a comprehensive information extraction module (STDC) is designed in the fusion network to enhance the network's ability to extract finer-grained complementary information from the source image. Finally, the fusion network and segmentation network are jointly trained to utilize semantic loss to guide the semantic information back to the fusion network, which effectively improves the performance of the fused images on high-level vision tasks. Extensive experiments show that our method is more effective than other state-of-the-art image fusion methods. In particular, our fused images not only have excellent visual perception effects, but also help to improve the performance of high-level vision tasks.

Unsupervised Semantic Segmentation with Feature Fusion

Research of improving semantic image segmentation based on a feature fusion model

Unsupervised segmentation via semantic-apparent feature fusion

Enhancing Feature Fusion with Spatial Aggregation and Channel Fusion for Semantic Segmentation

Multi-layer Adaptive Feature Fusion for Semantic Segmentation

ExFuse: Enhancing Feature Fusion for Semantic Segmentation

Semantic-Aware Fusion Network Based on Super-Resolution

MFEAFN: Multi-scale feature enhanced adaptive fusion network for image semantic segmentation

Stair Fusion Network for Remote Sensing Image Semantic Segmentation

Dual-Path Multi-Level Feature Fusion Network for Semantic Segmentation of Remote Sensing Image

Context and Boundary Guided Multi-Scale Feature Fusion Network for Semantic Segmentation

Semantic Segmentation of Remote-Sensing Images Based on Multiscale Feature Fusion and Attention Refinement

Weakly supervised point cloud semantic segmentation with the fusion of heterogeneous network features

Deep Feature Selection-And-Fusion for RGB-D Semantic Segmentation

A Multi-phase Camera-LiDAR Fusion Network for 3D Semantic Segmentation with Weak Supervision

Semantic Segmentation Based on Spatial Pyramid Pooling and Multilayer Feature Fusion

Feature Fusion Network Based on Hybrid Attention for Semantic Segmentation

A Multi-Step Fusion Network for Semantic Segmentation of High-Resolution Aerial Images

Enhanced Feature Pyramid Network for Semantic Segmentation.

Remote Sensing Image Semantic Segmentation Network Based on Multi-Scale Feature Enhancement Fusion

Multi-feature Fusion Network for Road Scene Semantic Segmentation.