Abstract:The current convolution-based semantic segmentation network lacks long-range dependencies for infrared small target detection, which may lead to unsatisfactory detection results in the real scenario. To address the problem, this paper proposed a semantic segmentation network based on hetero-range feature fusion (HRFFnet). Compared with the common semantic segmentation networks, this network includes two feature extraction branches. One is a short-range extraction branch consisting of convolution operations, and the other is a long-range feature extraction branch consisting of transformer. The HRFFnet complements the local features extracted by the convolutional neural network by adding the transformer structure to the segmentation network to introduce the long-range information of image. And this paper also designed a hetero-range fusion module. This module is based on atrous spatial pyramid pooling and adds shortcut connection to fuse different ranges of features extracted from images, which can maintain resolution of image and improve ability of feature representation. The hetero-range fusion module fuses long-range dependencies and short-range information extracted by transformer and convolution to capture multi-scale contextual information about the scene in infrared images and facilitate the interchange of information between different features. To evaluate the HRFFnet, we compare the performance of our network against other high-performance convolution-based methods and transformer-based networks on the open SIRST dataset with different evaluation metrics. The proposed method achieves a better combined results in terms of intersection of Dice coefficient, pixel accuracy, intersection over union and receiver operating characteristic curve. The experiments and results show that the network has some superiority: one is that it can break through the limitation of range of extracting features when only using convolutional network or transformer-based network; the other one is that this network can perform better with good robustness against real and complex scenarios. So, the proposed algorithm has broad application prospects in border patrol and urban security fields.

MS-AFF: a novel semantic segmentation approach for buried object based on multi-scale attentional feature fusion

Deep Dual-Stream Network with Scale Context Selection Attention Module for Semantic Segmentation

Lightweight Spatial Sliced-Concatenate-Multireceptive-Field Enhance and Joint Channel Attention Mechanism for Infrared Object Detection

Multi-Modal Object Detection Method Based on Dual-Branch Asymmetric Attention Backbone and Feature Fusion Pyramid Network

A Multi-Stage Visible and Infrared Image Fusion Network Based on Attention Mechanism

Multi-scale feature fusion attention network for infrared small target detection

A Multi-Task Framework for Infrared Small Target Detection and Segmentation

Infrared Maritime Object Detection Network With Feature Enhancement and Adjacent Fusion

MSCS: Multi-stage Feature Learning with Channel-Spatial Attention Mechanism for Infrared and Visible Image Fusion

DEAR: a novel deep-level semantics feature reinforce framework for Infrared Small Object Segmentation

Infrared and Visible Image Object Detection via Focused Feature Enhancement and Cascaded Semantic Extension

A Semantic-Aware and Multi-Guided Network for Infrared-Visible Image Fusion

Multi-scale unsupervised network for infrared and visible image fusion based on joint attention mechanism

Object recognition for millimeter wave SAR images based on dual-branch multi-scale fusion network

Multi-Scale Direction-Aware Network for Infrared Small Target Detection

Object Segmentation by Mining Cross-Modal Semantics

Infrared Small Target Detection based on Adjustable Sensitivity Strategy and Multi-Scale Fusion

Aware Distribute and Sparse Network for Infrared Small Target Detection

A Multi-phase Camera-LiDAR Fusion Network for 3D Semantic Segmentation with Weak Supervision

Multi-Sem Fusion: Multimodal Semantic Fusion for 3D Object Detection

Infrared small target detection algorithm based on hetero-range feature fusion