Abstract:With the rapid growth in demand for security surveillance, assisted driving, and remote sensing, object detection networks with robust environmental perception and high detection accuracy have become a research focus. However, single-modality image detection technologies face limitations in environmental adaptability, often affected by factors such as lighting conditions, fog, rain, and obstacles like vegetation, leading to information loss and reduced detection accuracy. We propose an object detection network that integrates features from visible light and infrared images—IV-YOLO—to address these challenges. This network is based on YOLOv8 (You Only Look Once v8) and employs a dual-branch fusion structure that leverages the complementary features of infrared and visible light images for target detection. We designed a Bidirectional Pyramid Feature Fusion structure (Bi-Fusion) to effectively integrate multimodal features, reducing errors from feature redundancy and extracting fine-grained features for small object detection. Additionally, we developed a Shuffle-SPP structure that combines channel and spatial attention to enhance the focus on deep features and extract richer information through upsampling. Regarding model optimization, we designed a loss function tailored for multi-scale object detection, accelerating the convergence speed of the network during training. Compared with the current state-of-the-art Dual-YOLO model, IV-YOLO achieves mAP improvements of 2.8%, 1.1%, and 2.2% on the Drone Vehicle, FLIR, and KAIST datasets, respectively. On the Drone Vehicle and FLIR datasets, IV-YOLO has a parameter count of 4.31 M and achieves a frame rate of 203.2 fps, significantly outperforming YOLOv8n (5.92 M parameters, 188.6 fps on the Drone Vehicle dataset) and YOLO-FIR (7.1 M parameters, 83.3 fps on the FLIR dataset), which had previously achieved the best performance on these datasets. This demonstrates that IV-YOLO achieves higher real-time detection performance while maintaining lower parameter complexity, making it highly promising for applications in autonomous driving, public safety, and beyond.

REQ-YOLO

Q-YOLO: Efficient Inference for Real-time Object Detection

An Efficient Real-Time Object Detection Framework on Resource-Constricted Hardware Devices via Software and Hardware Co-design

DAMO-YOLO : A Report on Real-Time Object Detection Design

YOLOv10: Real-Time End-to-End Object Detection

YOLO-DCNet

Edge Real-Time Object Detection and DPU-Based Hardware Implementation for Optical Remote Sensing Images

AQD: Towards Accurate Quantized Object Detection

DRBD-YOLOv8: A Lightweight and Efficient Anti-UAV Detection Model

LeYOLO, New Scalable and Efficient CNN Architecture for Object Detection

Reduced-Parameter YOLO-Like Object Detector Oriented to Resource-Constrained Platform

HV-YOLOv8 by HDPconv: Better lightweight detectors for small object detection

An FPGA-Based YOLOv5 Accelerator for Real-Time Industrial Vision Applications

A YOLO-NL object detector for real-time detection

A dedicated hardware accelerator for real-time acceleration of YOLOv2

YOLOv4-5D: An Effective and Efficient Object Detector for Autonomous Driving

FPGA-Based Vehicle Detection and Tracking Accelerator

YOLObile: Real-Time Object Detection on Mobile Devices via Compression-Compilation Co-Design

Octave-YOLO: Cross frequency detection network with octave convolution

Dq-YOLOF: An Effective Improvement with Deformable Convolution and Sample Quality Optimization Based on the YOLOF Detector

IV-YOLO: A Lightweight Dual-Branch Object Detection Network