Abstract:The Visible-Infrared (VIS-IR) object detection is a challenging detection task, which combines visible and infrared data to give information on the category and location of objects in the scene. Therefore, the core of this task is to combine complementary information in the visible and infrared modalities to provide more object detection results for detection. The existing methods mainly face the problem of insufficient ability to perceive and combine visible-infrared modal information and have difficulty in balancing the optimization directions of the fusion and detection tasks. To solve these problem, we propose the MMI-Det which is a multi-modal fusion method for visible and infrared object detection. The method can provide a good combination of complementary information in the visible-infrared modalities and output accurate and robust object information. Specifically, to improve the ability of the model to perceive environment at the visible-infrared image level, we designed the Contour Enhancement Module. Furthermore, to extract complementary information from VIS and IR modalities, we design the Fusion Focus Module. It can extract different frequency spectral features of the visible and infrared modalities and focus on the key information of the object at different spatial locations. Moreover, we design the Contrast Bridge Module to improve the ability to extract modal invariant features in the visible-infrared scene. Finally, to ensure that our model can balance the optimization directions of image fusion and object detection, we design the Info Guided Module as a way to improve the effectiveness of the model’s training optimization. We implement extensive experiments on the public FLIR, M3FD, LLVIP, TNO and MSRS datasets, and compared with previous methods, our method achieves better performance with powerful multi-modal information perception capabilities.

Multiview Infrared Target Detection and Localization

ATargetRecognitionMethodBasedonMultiview Infrared Images

Visible and Infrared Tracking Based on Multi-View Multi-Kernel Fusion Model

MMI-Det: Exploring Multi-Modal Integration for Visible and Infrared Object Detection

Local Information Guided Global Integration for Infrared Small Target Detection.

Multi-view 3-D Target Recognition Algorithm Based on Moment Invariant and Support Vector Machine Classification

Infrared Small Target Detection Using Local and Nonlocal Spatial Information.

Infrared Dim and Small Target Detection Based on Local–Global Feature Fusion

Infrared Small Target Detection Based on Multidirectional Gradient.

Small Infrared Target Fusion Detection Based on Support Vector Machines in the Wavelet Domain

IR Target Detection Via Lateral Inhibition and Singular Value Decomposition

Multiview Fusion 3D Target Information Perception Model in Nighttime Unmanned Intelligent Vehicles

Infrared Target Detection And Location For Visual Surveillance Using Fusion Scheme Of Visible And Infrared Images

Biologically Inspired Multilevel Approach for Multiple Moving Targets Detection from Airborne Forward-Looking Infrared Sequences.

Multifeature distance map based detection of small infrared targets with low contrast in Image sequences

Infrared Small-Target Detection Based on Multi-Directional Multi-Scale High-Boost Response

VIMI: Vehicle-Infrastructure Multi-view Intermediate Fusion for Camera-based 3D Object Detection

Infrared Small Target Detection Based on Local Significance and Multiscale

Robust Night Target Tracking Via Infrared and Visible Video Fusion

Target Detection and Tracking in Infrared Video

Multi-view Target Recognition Algorithm Based on Support Vector Machine Classification