Abstract:With the development of unmanned aerial vehicles (UAVs) and computer vision, target detection methods based on UAVs have been increasingly applied in military and civilian fields. Considering the adaptability requirements of low illumination environments such as rain, fog, and night, visible and infrared (IR) sensors are often installed on UAVs to perform in all-weather and all-day conditions. To improve the near-surface detection performance of UAVs in low illumination environments, a pedestrian detection method using image fusion and deep learning is proposed. Visible and IR pedestrian images are collected by the UAV. The corresponding aerial images are registered and annotated. These two different types of aerial images are aligned at the time sequence and matched using the scale invariant feature transform. A U-type generative adversarial network (GAN) is first developed to fuse visible and IR images. A convolutional block attention module is introduced to strengthen the pedestrian target information in the GAN. The spatial domain and channel domain attention mechanisms are proposed to generate color fusion images with rich details and solve the problems of feature extraction as well as fusion rules designed manually in the existing image fusion methods. Then, You Only Look Once Version 3 (YOLOv3)-spatial pyramid pooling combined with transfer learning is adopted using the fused images to train the model on our aerial dataset to verify the pedestrian detection performance. In addition, comparison experiments are carried out. The experimental results demonstrate that the YOLOv3 model is successfully transferred to the target dataset. The performance of the proposed detection model using the fused images for transfer training is the best among the different methods. Finally, the accuracy P, recall R, mean average precision, and F-1 score reach 0.804, 0.923, 0.928, and 0.859, respectively. (C) 2022 Society of Photo-Optical Instrumentation Engineers (SPIE)

Application of image-to-image translation in improving pedestrian detection

Pedestrian Detection Method in Infrared Image Based on Improved YOLOv7

Let There be Light: Improved Traffic Surveillance via Detail Preserving Night-to-Day Transfer

Pedestrain detection for low-light vision proposal

Pedestrian Partial Occlusion Detection Method Based on Improved YOLO Network Structure

Low-light Pedestrian Detection in Visible and Infrared Image Feeds: Issues and Challenges

Research on Multi-Modal Pedestrian Detection and Tracking Algorithm Based on Deep Learning

Overview of Pedestrian Detection Based on Infrared Image

A Novel Approach to Pedestrian Re-Identification in Low-Light and Zero-Shot Scenarios: Exploring Transposed Convolutional Reflectance Decoders

TF-YOLO: A Transformer–Fusion-Based YOLO Detector for Multimodal Pedestrian Detection in Autonomous Driving Scenes

YOLOpeds: Efficient Real-Time Single-Shot Pedestrian Detection for Smart Camera Applications

Learning Cross-Modal Deep Representations for Robust Pedestrian Detection

Multispectral Pedestrian Detection Via Two-Stream YOLO With Complementarity Fusion For Autonomous Driving

A Method of Infrared Image Pedestrian Detection with Improved YOLOv3 Algorithm

Pedestrian Detection in Low-Light Conditions: A Comprehensive Survey

The Improved Pedestrian Target Detection and Re-recognition Algorithm

Pedestrian Detection: Domain Generalization, CNNs, Transformers and Beyond

Near-surface Pedestrian Detection Method Based on Deep Learning for UAVs in Low Illumination Environments

Illumination and Temperature-Aware Multispectral Networks for Edge-Computing-Enabled Pedestrian Detection

GAN-Based Day-to-Night Image Style Transfer for Nighttime Vehicle Detection

Robust Pedestrian Detection Based on Multi-Spectral Image Fusion and Convolutional Neural Networks