Abstract:Visible-infrared object detection for remote sensing images plays an important role in the unmanned aerial vehicle (UAV) around-the-clock application. Most of the existing work focuses on designing complex network architectures to fuse complementary features, while few methods consider computational complexity and susceptibility against modality attacks, limiting the deployment of state-of-the-art frameworks. In this article, we present a low-rank multimodal object detection approach with frequency filtering experts, called LF-MDet, which is based on the advanced DINO (detection transformer with improved denoising anchor boxes) framework. This approach achieves more accurate detection with fewer computational resources. In particular, when the specific modality is attacked or missing, our method still maintains higher robustness toward such pervasive perturbations. Specifically, we propose a low-rank enhancement technology (LET) and a dynamic illumination-aware mask (DIM) module to enable a single backbone network in the form of batch formulation to unbiasedly and compatibly extract multimodal features. Furthermore, we design a lightweight frequency expert encoder (FEE) from the frequency domain perspective to efficiently fuse complementary features by filtering out amplitude noise components and mixing feature tokens. Extensive experiments are conducted on the multimodal remote sensing object detection datasets, VEDAI and DroneVehicle. The results demonstrate the superiority of the proposed approach over advanced multimodal remote sensing object detectors. Compared to the baseline method, our low-rank multimodal detector (LF-MDet) effectively reduces the floating point of operations (FLOPs) by approximately 65% while improving detection accuracy. The code is available at https://github.com/cq100/LF-MDet.

Adaptive multimodal feature fusion with frequency domain gate for remote sensing object detection

Adaptive Switching Spatial-Temporal Fusion Detection for Remote Flying Drones

Multi-spectral Image Fusion for Moving Object Detection

Dual Stream Cross Domain Feature Fusion for Land-Oceanic Object Detection

Low-Rank Multimodal Remote Sensing Object Detection With Frequency Filtering Experts

An object detection algorithm based on infrared-visible dual modal feature fusion

Multi-modal Object Detection of UAV Remote Sensing Based on Joint Representation Optimization and Specific Information Enhancement

Multi-Modal Object Detection Method Based on Dual-Branch Asymmetric Attention Backbone and Feature Fusion Pyramid Network

ICAFusion: Iterative cross-attention guided feature fusion for multispectral object detection

An Adaptive Attention Fusion Mechanism Convolutional Network for Object Detection in Remote Sensing Images

Multi-Modal Target Detection Method Based on Adaptive Feature Search

DFECF-DET: All-Weather Detector Based on Differential Feature Enhancement and Cross-Modal Fusion With Visible and Infrared Sensors

Enhanced Spectral-Spatial Fusion Network for Multispectral Object Detection in Ground-Aerial Images

Cross-Modality Attentive Feature Fusion for Object Detection in Multispectral Remote Sensing Imagery

DMFF: dual-way multimodal feature fusion for 3D object detection

ACDF-YOLO: Attentive and Cross-Differential Fusion Network for Multimodal Remote Sensing Object Detection

Airborne Small Target Detection Method Based on Multimodal and Adaptive Feature Fusion

Mask-Guided Mamba Fusion for Drone-Based Visible-Infrared Vehicle Detection

Multispectral Object Detection Based on Multilevel Feature Fusion and Dual Feature Modulation

MMDR: A Result Feature Fusion Object Detection Approach for Autonomous System

Multi-Modal Fusion Based on Depth Adaptive Mechanism for 3D Object Detection