Abstract:With the rapid growth in demand for security surveillance, assisted driving, and remote sensing, object detection networks with robust environmental perception and high detection accuracy have become a research focus. However, single-modality image detection technologies face limitations in environmental adaptability, often affected by factors such as lighting conditions, fog, rain, and obstacles like vegetation, leading to information loss and reduced detection accuracy. We propose an object detection network that integrates features from visible light and infrared images—IV-YOLO—to address these challenges. This network is based on YOLOv8 (You Only Look Once v8) and employs a dual-branch fusion structure that leverages the complementary features of infrared and visible light images for target detection. We designed a Bidirectional Pyramid Feature Fusion structure (Bi-Fusion) to effectively integrate multimodal features, reducing errors from feature redundancy and extracting fine-grained features for small object detection. Additionally, we developed a Shuffle-SPP structure that combines channel and spatial attention to enhance the focus on deep features and extract richer information through upsampling. Regarding model optimization, we designed a loss function tailored for multi-scale object detection, accelerating the convergence speed of the network during training. Compared with the current state-of-the-art Dual-YOLO model, IV-YOLO achieves mAP improvements of 2.8%, 1.1%, and 2.2% on the Drone Vehicle, FLIR, and KAIST datasets, respectively. On the Drone Vehicle and FLIR datasets, IV-YOLO has a parameter count of 4.31 M and achieves a frame rate of 203.2 fps, significantly outperforming YOLOv8n (5.92 M parameters, 188.6 fps on the Drone Vehicle dataset) and YOLO-FIR (7.1 M parameters, 83.3 fps on the FLIR dataset), which had previously achieved the best performance on these datasets. This demonstrates that IV-YOLO achieves higher real-time detection performance while maintaining lower parameter complexity, making it highly promising for applications in autonomous driving, public safety, and beyond.

YOLO-CIR: The network based on YOLO and ConvNeXt for infrared object detection

A Lightweight SE-YOLOv3 Network for Multi-Scale Object Detection in Remote Sensing Imagery.

Dual-YOLO Architecture from Infrared and Visible Images for Object Detection

IV-YOLO: A Lightweight Dual-Branch Object Detection Network

Dual-branch network object detection algorithm based on dual-modality fusion of visible and infrared images

Infra-YOLO: Efficient Neural Network Structure with Model Compression for Real-Time Infrared Small Object Detection

LFIR-YOLO: Lightweight Model for Infrared Vehicle and Pedestrian Detection

Infrared Target Detection Based on the Fusion of Attention Mechanism and YOLOv5

An Object Detection Method Based on Improved YOLOX

Improved lightweight infrared road target detection method based on YOLOv8

FA-YOLO: An Improved YOLO Model for Infrared Occlusion Object Detection under Confusing Background

Research on Infrared Dim Target Detection Based on Improved YOLOv8

Lightweight Spatial Sliced-Concatenate-Multireceptive-Field Enhance and Joint Channel Attention Mechanism for Infrared Object Detection

YOLO-B:An infrared target detection algorithm based on bi-fusion and efficient decoupled

Target Recognition Based on Infrared and Visible Image Fusion and Improved YOLOv8 Algorithm

A Convolution with Transformer Attention Module Integrating Local and Global Features for Object Detection in Remote Sensing Based on YOLOv8n

YOLOSR-IST: A Deep Learning Method for Small Target Detection in Infrared Remote Sensing Images based on Super-Resolution and YOLO

An object detection algorithm based on infrared-visible dual modal feature fusion

Infrared and Visible Image Object Detection via Focused Feature Enhancement and Cascaded Semantic Extension

Infrared Dim and Small Target Detection Based on Local–Global Feature Fusion

YOLOFM: an improved fire and smoke object detection algorithm based on YOLOv5n