Abstract:Multi-sensor information fusion technology has been widely used in the perception of unmanned aerial vehicle environments. However, the perception accuracy needs to be improved in practice since multiple sensors have consistency limitations and fused data has limited utility. A deep learning method based on the multi-stage fusion of millimeter-wave radar and camera is proposed in this paper. In the data preprocessing stage, the radar reflection point and image pixels are fused in a Gaussian-weighted way to obtain the salient image. The salient density map of each pixel relative to the radar reflection point is calculated. Then, the threshold is set to segment the salient density map to complete visual target detection. In the detection stage based on deep learning, a network structure is designed to fuse the salient image and visual target detection images at different convolution depths. The classification, location, and size of targets are regressed by training. In the post-decoding stage, the radar reflection point is fused for local non-maximum suppression. The non-maximum suppression operation is started from the radar reflection point. Different from typical detection methods, the proposed method improves detection accuracy by fusing the feature information of the radar and camera in a multi-stage process. Experimental results demonstrated that mAP0.90 increased by 3.9% and 4.3%. For complex scenarios, mAP0.50 improved by 2.4%, mAP0.75 improved by 4.9%, and mAP0.90 improved by 6.9%, indicating that the proposed method is effective compared to the state-of-the-art model (YOLOv8).

A High-resolution Radar Automatic Target Recognition Method for Small UAVs Based on Multi-feature Fusion

Realizing Small UAV Targets Recognition via Multi-Dimensional Feature Fusion of High-Resolution Radar

Fast Detection and Recognition Method of UAV in Sky Background

A Small UAV Detection Method Based on Optical Flow and Visual Feature Fusion

Low in Resolution, High in Precision: UAV Detection with Super-Resolution and Motion Information Extraction

Detection and Recognition Method of Fast Low-Altitude Unmanned Aerial Vehicle Based on Dual Channel

Research on Information Fusion of Computer Vision and Radar Signals in UAV Target Identification

A Low-Slow-Small UAV Detection Method Based on Fusion of Range Doppler Map and Satellite Map

Recognition of flying small target based on enhanced quadratic time-frequency analysis

A Complete Automatic Target Recognition System of Low Altitude, Small RCS and Slow Speed (LSS) Targets Based on Multi-Dimensional Feature Fusion

UAV Target Identification Based on Information fusion of Computer Vision and Radar Signals

Multi-Target Tracking with Multiple Unmanned Aerial Vehicles Based on Information Fusion

High-precision real-time UAV target recognition based on improved YOLOv4

Aerial Target Recognition with Enhanced Micro-Doppler Dynamic Features Based on Frequency-Modulated Continuous-Wave Radar

Multi-band Image Fusion with Infrared Broad Spectrum for Low and Slow Small Target Recognition

Multi-scale object detection in UAV images based on adaptive feature fusion

Multilevel Recognition of UAV-to-Ground Targets Based on Micro-Doppler Signatures and Transfer Learning of Deep Convolutional Neural Networks

UAV Classification Based on Deep Learning Fusion of Multidimensional UAV Micro-Doppler Image Features

Fusion of Millimeter Wave Radar and Monocular Vision for Obstacle Detection Based on Rotor-Wing UAV

Study of Multi-View UAV Target Recognition

A Deep Learning Method Based on the Multi-stage Fusion of Radar and Camera in UAV Obstacle Avoidance