Abstract:Infrared and visible images (multi-sensor or multi-band images) have many complementary features which can effectively boost the performance of object detection. Recently, convolutional neural networks (CNNs) have seen frequent use to perform object detection in multi-band images. However, it is very difficult for CNNs to extract complementary features from infrared and visible images. In order to solve this problem, a difference maximum loss function is proposed in this paper. The loss function can guide the learning directions of two base CNNs and maximize the difference between features from the two base CNNs, so as to extract complementary and diverse features. In addition, we design a focused feature-enhancement module to make features in the shallow convolutional layer more significant. In this way, the detection performance of small objects can be effectively improved while not increasing the computational cost in the testing stage. Furthermore, since the actual receptive field is usually much smaller than the theoretical receptive field, the deep convolutional layer would not have sufficient semantic features for accurate detection of large objects. To overcome this drawback, a cascaded semantic extension module is added to the deep layer. Through simple multi-branch convolutional layers and dilated convolutions with different dilation rates, the cascaded semantic extension module can effectively enlarge the actual receptive field and increase the detection accuracy of large objects. We compare our detection network with five other state-of-the-art infrared and visible image object detection networks. Qualitative and quantitative experimental results prove the superiority of the proposed detection network.

Fusion of infrared and visible light images for object detection based on CNN

Fusion of Low-Illuminance Visible and Near-Infrared Images Based on Convolutional Neural Networks

HDCTfusion: Hybrid Dual-Branch Network Based on CNN and Transformer for Infrared and Visible Image Fusion

Infrared and Visible Image Fusion with Convolutional Neural Networks.

Infrared and Visible Image Object Detection via Focused Feature Enhancement and Cascaded Semantic Extension

DCFusion: A Dual-Frequency Cross-Enhanced Fusion Network for Infrared and Visible Image Fusion.

An object detection algorithm based on infrared-visible dual modal feature fusion

FTC-Net: Fusion of Transformer and CNN Features for Infrared Small Target Detection

Dual-branch network object detection algorithm based on dual-modality fusion of visible and infrared images

Infrared and Visible Light Fusion for Object Detection with Low-light Enhancement

HDCCT: Hybrid Densely Connected CNN and Transformer for Infrared and Visible Image Fusion

Multi-scale Convolutional Neural Networks and Saliency Weight Maps for Infrared and Visible Image Fusion

An Adaptive Attention Fusion Mechanism Convolutional Network for Object Detection in Remote Sensing Images

LatLRR-CNN: an Infrared and Visible Image Fusion Method Combining Latent Low-Rank Representation and CNN.

A Multi-Branch Feature Fusion Network for Building Detection in Remote Sensing Images

HATF: Multi-Modal Feature Learning for Infrared and Visible Image Fusion via Hybrid Attention Transformer

Infrared and Visible Images Fusion Using Visual Saliency and Dual-PCNN

HCF-Net: Hierarchical Context Fusion Network for Infrared Small Object Detection

A Multispectral-Infrared Object Detection Method Based on Cross-Modality Image Feature Filtering Fusion

Infrared and visible image fusion with entropy-based adaptive fusion module and mask-guided convolutional neural network

Visible and Infrared Image Fusion Based on Attention and Multiscale Residuals