Abstract:Arbitrary-oriented object detection is vital for improving unmanned aerial vehicle (UAV) sensing and has promising applications. However, challenges persist in detecting objects under extreme conditions like low-illumination and strong occlusion. Cross-modal feature fusion enhances detection in complex environments but current methods do not adequately learn the features of each modality for the current environment, resulting in degraded performance. To tackle this, we propose the cross-modal aerial remote sensing image object detection (CRSIOD) network that effectively learns diverse sensor image features to capture distinct scenarios and target characteristics. First, we design an illumination perception module to guide the object detection network in performing various feature processing tasks. Second, to leverage the respective advantages of two modalities and mitigate their negative impacts, we introduce an uncertainty-aware module to quantify the uncertainties present in each modality as weights to motivate the network to learn in a direction favorable for optimal object detection. Moreover, in the object detection network, we design a two-stream backbone network based on the attention mechanism to enhance the learning of difficult samples, utilize the cross-modality attentive feature fusion (CMAFF) module to fully extract the shared and complementary features between the two modalities, and design a three-branch feature enhancement network to enhance the learning of the three modal features separately. Finally, to optimize detection results, we design light perception nonmaximum suppression (LP-NMS) and improve the horizontal detection head to a rotating one to preserve object orientation. We evaluate the proposed method CRSIOD on the Drone Vehicle dataset of public UAV aerial images. Compared with the existing commonly used methods, CRSIOD achieves state-of-the-art detection performance.

Salient Object Detection with Bayesian Inference Based on Radar and Camera Fusion Used in UAV Obstacle Avoidance

Salient Object Detection with High-Level Prior Based on Bayesian Fusion.

A Deep Learning Method Based on the Multi-stage Fusion of Radar and Camera in UAV Obstacle Avoidance

Radar-Optical Fusion Detection of UAV Based On Improved YOLOv7-tiny

ROFusion: Efficient Object Detection using Hybrid Point-wise Radar-Optical Fusion

Spatial Attention Fusion for Obstacle Detection Using MmWave Radar and Vision Sensor

Maritime Target Saliency Detection for UAV Based on the Stimulation Competition Selection Mechanism of Raptor Vision

Efficient Fourier Filtering Network with Contrastive Learning for UAV-based Unaligned Bi-modal Salient Object Detection

Fusion of Millimeter Wave Radar and Monocular Vision for Obstacle Detection Based on Rotor-Wing UAV

Robust Salient Object Detection Via Fusing Foreground and Background Priors.

RGB-D Salient Object Detection Method Based on Multi-Modal Fusion and Contour Guidance

New Contour Cue-Based Hybrid Sparse Learning for Salient Object Detection

Radar-Camera Sensor Fusion for Joint Object Detection and Distance Estimation in Autonomous Vehicles

Robust Small Object Detection on the Water Surface Through Fusion of Camera and Millimeter Wave Radar

Radar Guided Dynamic Visual Attention for Resource-Efficient RGB Object Detection

Cross-Modal Oriented Object Detection of UAV Aerial Images Based on Image Feature

Fast Detection and Obstacle Avoidance on UAVs Using Lightweight Convolutional Neural Network Based on the Fusion of Radar and Camera

UAV-Based Human Body Detector Selection and Fusion for Geolocated Saliency Map Generation

A novel biologically-inspired target detection method based on saliency analysis for synthetic aperture radar (SAR) imagery

Research on UAV Obstacle Detection Based on Data Fusion of Millimeter Wave Radar and Monocular Camera

Radar-Lidar Fusion for Object Detection by Designing Effective Convolution Networks