Abstract:This study delves into the problem of moving object detection in infrared and visible images. While existing approaches primarily focus on single-task detection using single-spectral image data, such as thermal infrared or visible images, they often ignore the differences between different spectral images and the interconnectedness of related tasks such as image fusion and segmentation. To tackle these problems, we present a novel multi-spectral image fusion network with quality and semantic awareness for moving object detection (MOD), particularly in scenarios where ground truth labels for both infrared and visible images are unavailable. Our approach fuses multispectral images, incorporating additional subtasks in the image fusion to obtain content and quality perception of infrared and visible images. In addition, we design a novel residual global perception module (RGPM) and multi-spectral fusion loss, which can capture more hidden features and contextual information across various scales. This enhanced capability leads to more precise detection and tracking of moving objects, particularly in challenging situations involving occlusions and dynamic backgrounds. Compared with single-spectral moving object detection optimization, the hurdles of utilizing deep learning for multi-spectral image fusion, e.g., without ground truth labels and harmful noise, are significantly mitigated. Extensive quantitative and qualitative comparative experiments demonstrate its effectiveness, robustness, and superior performance compared to contemporary methods. Concisely, the proposed fusion representation learning has gained 44.2%,31.2%,36.3%,41.6%,5.3%,31.2%,76.2% on EI, SF, DF, AG, MI, SD and Nabf metrics compared with the best competitors.

Multi-feature fusion based GMM for moving object and shadow detection

Shadow Removal Based on Multi-Feature Fusion of Multiple Moving Objects Scene

Moving shadow detection using fusion of multiple features

A Moving Shadow Elimination Method Based on Fusion of Multi-Feature

Moving Shadow Detection by Integrating Multiple Features

Shadow Detection with Multi-Feature Fusion and MMSE Optimization

An Algorithm of Motion Detection Based on the Multi-Feature Background Model

Adaptive Shadow Detection Based on GMM and MRF

Moving Cast Shadow Suppression from a Gaussian Mixture Shadow Model

A Fast Convergent Gaussian Mixture Model in Moving Object Detection with Shadow Elimination

Moving Shadow Detection Based on Gray and Texture Features

Integration of Multi-feature for Moving Target Detection Algorithm

An Effective Moving Cast Shadow Detection Framework for Surveillance Video Analysis

Improved Mixed Gaussian Model for Background Subtraction Based on Color Channel Fusion

Shadow Removal Algorithm by Combining Gradient of the Brightness Ratio and Color Fea-tures

Moving object detection based on improved Gaussian Mixture Models

Moving Object Detection and Shadow Removing under Changing Illumination Condition

Multi-spectral Image Fusion for Moving Object Detection

Video Segmentation Algorithm with Gaussian Mixture Model and Shadow Removal

Moving Objects Detection Method Based on the Improved Adaptive Background Mixture Model

Multimodal background model with noise and shadow suppression for moving object detection