Abstract:This study delves into the problem of moving object detection in infrared and visible images. While existing approaches primarily focus on single-task detection using single-spectral image data, such as thermal infrared or visible images, they often ignore the differences between different spectral images and the interconnectedness of related tasks such as image fusion and segmentation. To tackle these problems, we present a novel multi-spectral image fusion network with quality and semantic awareness for moving object detection (MOD), particularly in scenarios where ground truth labels for both infrared and visible images are unavailable. Our approach fuses multispectral images, incorporating additional subtasks in the image fusion to obtain content and quality perception of infrared and visible images. In addition, we design a novel residual global perception module (RGPM) and multi-spectral fusion loss, which can capture more hidden features and contextual information across various scales. This enhanced capability leads to more precise detection and tracking of moving objects, particularly in challenging situations involving occlusions and dynamic backgrounds. Compared with single-spectral moving object detection optimization, the hurdles of utilizing deep learning for multi-spectral image fusion, e.g., without ground truth labels and harmful noise, are significantly mitigated. Extensive quantitative and qualitative comparative experiments demonstrate its effectiveness, robustness, and superior performance compared to contemporary methods. Concisely, the proposed fusion representation learning has gained 44.2%,31.2%,36.3%,41.6%,5.3%,31.2%,76.2% on EI, SF, DF, AG, MI, SD and Nabf metrics compared with the best competitors.

SiamMMF: multi-modal multi-level fusion object tracking based on Siamese networks

DSiamMFT: An RGB-T fusion tracking method via dynamic Siamese networks using multi-layer feature fusion

Siamese Network with Multi-Scale Fusion Attention for Visual Tracking

SiamFT: an RGB-Infrared Fusion Tracking Method Via Fully Convolutional Siamese Networks

MHASiam: Mixed High-Order Attention Siamese Network for Real-Time Visual Tracking

Adaptive Multi-Feature Fusion Visual Target Tracking Based on Siamese Neural Network with Cross-Attention Mechanism

Siamese infrared and visible light fusion network for RGB-T tracking

Object Fusion Tracking Based on Visible and Infrared Images Using Fully Convolutional Siamese Networks

Mutual Learning and Feature Fusion Siamese Networks for Visual Object Tracking

MFST: Multi-Features Siamese Tracker

SiamDA: Dual Attention Siamese Network for Real-Time Visual Tracking

SiamCAF: Complementary Attention Fusion-Based Siamese Network for RGBT Tracking

MMF-Track: Multi-modal Multi-level Fusion for 3D Single Object Tracking

Multiple Convolutional Features in Siamese Networks for Object Tracking

Learning Siamese Network with Top-Down Modulation for Visual Tracking

Multi-spectral Image Fusion for Moving Object Detection

Multi-scale Feature Extraction and Fusion with Attention Interaction for RGB-T Tracking

Infrared Object Tracking Algorithm with Temporal Features Fusion Siamese Network

Multitarget Tracking Using Siamese Neural Networks

Siamese Tracking Network with Multi-attention Mechanism

Deep Adaptive Discriminate Siamese Network with Multi-Level Response for Visual Object Tracking