Abstract:With the rapid growth in demand for security surveillance, assisted driving, and remote sensing, object detection networks with robust environmental perception and high detection accuracy have become a research focus. However, single-modality image detection technologies face limitations in environmental adaptability, often affected by factors such as lighting conditions, fog, rain, and obstacles like vegetation, leading to information loss and reduced detection accuracy. We propose an object detection network that integrates features from visible light and infrared images—IV-YOLO—to address these challenges. This network is based on YOLOv8 (You Only Look Once v8) and employs a dual-branch fusion structure that leverages the complementary features of infrared and visible light images for target detection. We designed a Bidirectional Pyramid Feature Fusion structure (Bi-Fusion) to effectively integrate multimodal features, reducing errors from feature redundancy and extracting fine-grained features for small object detection. Additionally, we developed a Shuffle-SPP structure that combines channel and spatial attention to enhance the focus on deep features and extract richer information through upsampling. Regarding model optimization, we designed a loss function tailored for multi-scale object detection, accelerating the convergence speed of the network during training. Compared with the current state-of-the-art Dual-YOLO model, IV-YOLO achieves mAP improvements of 2.8%, 1.1%, and 2.2% on the Drone Vehicle, FLIR, and KAIST datasets, respectively. On the Drone Vehicle and FLIR datasets, IV-YOLO has a parameter count of 4.31 M and achieves a frame rate of 203.2 fps, significantly outperforming YOLOv8n (5.92 M parameters, 188.6 fps on the Drone Vehicle dataset) and YOLO-FIR (7.1 M parameters, 83.3 fps on the FLIR dataset), which had previously achieved the best performance on these datasets. This demonstrates that IV-YOLO achieves higher real-time detection performance while maintaining lower parameter complexity, making it highly promising for applications in autonomous driving, public safety, and beyond.

Target Detection Based on Two-Stream Convolution Neural Network With Self-Powered Sensors Information

Real-Time Target Detection Method Based on Lightweight Convolutional Neural Network

Lightweight Target Detection Algorithm Based on Adaptive Spatial Feature Fusion

Multi-Color Vehicle Tracking Based on Lightweight Neural Network

IV-YOLO: A Lightweight Dual-Branch Object Detection Network

MobileYOLO: Real-Time Object Detection Algorithm in Autonomous Driving Scenarios

YOLO-G: A Lightweight Network Model for Improving the Performance of Military Targets Detection

Lightweight target detection algorithm based on YOLOv4

Multiscale and Direction Target Detecting in Remote Sensing Images via Modified YOLO-v4

Real-time object detection method based on YOLOv5 and efficient mobile network

Gated Channel Attention Mechanism YOLOv3 Network for Small Target Detection

A New Method of Different Neural Network Depth and Feature Map Size on Remote Sensing Small Target Detection

Improved YOLO-V3 with DenseNet for Multi-Scale Remote Sensing Target Detection

Target detection for remote sensing based on the enhanced YOLOv4 with improved BiFPN

RD-YOLO Target Detection for Night Images

An Improved Mobilenetv3-Yolov5 Infrared Target Detection Algorithm Based on Attention Distillation

YOLO-MPAM: Efficient real-time neural networks based on multi-channel feature fusion

L-SSD: lightweight SSD target detection based on depth-separable convolution

High-precision real-time autonomous driving target detection based on YOLOv8