Abstract:With the rapid growth in demand for security surveillance, assisted driving, and remote sensing, object detection networks with robust environmental perception and high detection accuracy have become a research focus. However, single-modality image detection technologies face limitations in environmental adaptability, often affected by factors such as lighting conditions, fog, rain, and obstacles like vegetation, leading to information loss and reduced detection accuracy. We propose an object detection network that integrates features from visible light and infrared images—IV-YOLO—to address these challenges. This network is based on YOLOv8 (You Only Look Once v8) and employs a dual-branch fusion structure that leverages the complementary features of infrared and visible light images for target detection. We designed a Bidirectional Pyramid Feature Fusion structure (Bi-Fusion) to effectively integrate multimodal features, reducing errors from feature redundancy and extracting fine-grained features for small object detection. Additionally, we developed a Shuffle-SPP structure that combines channel and spatial attention to enhance the focus on deep features and extract richer information through upsampling. Regarding model optimization, we designed a loss function tailored for multi-scale object detection, accelerating the convergence speed of the network during training. Compared with the current state-of-the-art Dual-YOLO model, IV-YOLO achieves mAP improvements of 2.8%, 1.1%, and 2.2% on the Drone Vehicle, FLIR, and KAIST datasets, respectively. On the Drone Vehicle and FLIR datasets, IV-YOLO has a parameter count of 4.31 M and achieves a frame rate of 203.2 fps, significantly outperforming YOLOv8n (5.92 M parameters, 188.6 fps on the Drone Vehicle dataset) and YOLO-FIR (7.1 M parameters, 83.3 fps on the FLIR dataset), which had previously achieved the best performance on these datasets. This demonstrates that IV-YOLO achieves higher real-time detection performance while maintaining lower parameter complexity, making it highly promising for applications in autonomous driving, public safety, and beyond.

Multi-Object Robot Visual Servo Based on YOLOv3

Intelligent Perception System of Robot Visual Servo for Complex Industrial Environment

Visual Servoing with an Uncalibrated Eye-in-hand Camera

MGBM-YOLO: a Faster Light-Weight Object Detection Model for Robotic Grasping of Bolster Spring Based on Image-Based Visual Servoing

An Image Based Visual Servo Approach with Deep Learning for Robotic Manipulation

Siamese Convolutional Neural Network Based Visual Servo for Manipulator

An Object Detection Method Based on Improved YOLOX

Visual Servoing with Deep Learning and Data Augmentation for Robotic Manipulation

Keypoint Detection Technique for Image-Based Visual Servoing of Manipulators

YOLOv8-PoseBoost: Advancements in Multimodal Robot Pose Keypoint Detection

Object Detection Method for Grasping Robot Based on Improved YOLOv5

Motion Prediction and Object Detection for Image-Based Visual Servoing Systems Using Deep Learning

Robotic grasping method of bolster spring based on image-based visual servoing with YOLOv3 object detection algorithm

Siamese Convolutional Neural Network for Sub-millimeter-accurate Camera Pose Estimation and Visual Servoing

Hybrid Multi-camera Visual Servoing to Moving Target

A Two-Stage Monocular Vision Detection Method for 6D Pose Estimation in Multi-Heterogeneous Robot Systems

End-to-end Multi-Instance Robotic Reaching from Monocular Vision

Multi-YOLOv8: An infrared moving small object detection model based on YOLOv8 for air vehicle

IV-YOLO: A Lightweight Dual-Branch Object Detection Network

A Hyper-network Based End-to-end Visual Servoing with Arbitrary Desired Poses

Image-based Visual Servoing for Mobile Robots