Abstract:With the rapid growth in demand for security surveillance, assisted driving, and remote sensing, object detection networks with robust environmental perception and high detection accuracy have become a research focus. However, single-modality image detection technologies face limitations in environmental adaptability, often affected by factors such as lighting conditions, fog, rain, and obstacles like vegetation, leading to information loss and reduced detection accuracy. We propose an object detection network that integrates features from visible light and infrared images—IV-YOLO—to address these challenges. This network is based on YOLOv8 (You Only Look Once v8) and employs a dual-branch fusion structure that leverages the complementary features of infrared and visible light images for target detection. We designed a Bidirectional Pyramid Feature Fusion structure (Bi-Fusion) to effectively integrate multimodal features, reducing errors from feature redundancy and extracting fine-grained features for small object detection. Additionally, we developed a Shuffle-SPP structure that combines channel and spatial attention to enhance the focus on deep features and extract richer information through upsampling. Regarding model optimization, we designed a loss function tailored for multi-scale object detection, accelerating the convergence speed of the network during training. Compared with the current state-of-the-art Dual-YOLO model, IV-YOLO achieves mAP improvements of 2.8%, 1.1%, and 2.2% on the Drone Vehicle, FLIR, and KAIST datasets, respectively. On the Drone Vehicle and FLIR datasets, IV-YOLO has a parameter count of 4.31 M and achieves a frame rate of 203.2 fps, significantly outperforming YOLOv8n (5.92 M parameters, 188.6 fps on the Drone Vehicle dataset) and YOLO-FIR (7.1 M parameters, 83.3 fps on the FLIR dataset), which had previously achieved the best performance on these datasets. This demonstrates that IV-YOLO achieves higher real-time detection performance while maintaining lower parameter complexity, making it highly promising for applications in autonomous driving, public safety, and beyond.

SECOND: Sparsely Embedded Convolutional Detection

SSF: Sparse Point Cloud Object Detection Based on Self-Adaptive Voxel Encoding and Focal-Sparse Convolution

A Lightweight SE-YOLOv3 Network for Multi-Scale Object Detection in Remote Sensing Imagery.

Virtual Sparse Convolution for Multimodal 3D Object Detection

Sparse Embedded Convolution Based Dual Feature Aggregation 3D Object Detection Network

PVConvNet: Pixel-Voxel Sparse Convolution for multimodal 3D object detection

SparseDet: A Simple and Effective Framework for Fully Sparse LiDAR-based 3D Object Detection

Real-Time 3D Object Detection From Point Cloud Through Foreground Segmentation

LiDAR-only 3D Object Detection Based on Spatial Context

Semantic frustum-based sparsely embedded convolutional detection

SparseDet: A Simple and Effective Framework for Fully Sparse LiDAR-Based 3-D Object Detection

Sparse Points to Dense Clouds: Enhancing 3D Detection with Limited LiDAR Data

DVFENet: Dual-branch voxel feature extraction network for 3D object detection

Spatial and Semantic Information Enhancement for Indoor 3D Object Detection

Voxel or Pillar: Exploring Efficient Point Cloud Representation for 3D Object Detection

Efficient three-dimensional point cloud object detection based on improved Complex-YOLO

Context-Aware Dynamic Feature Extraction for 3D Object Detection in Point Clouds

IV-YOLO: A Lightweight Dual-Branch Object Detection Network

Sparse2Dense: Learning to Densify 3D Features for 3D Object Detection

PV-RCNN++: Semantical Point-Voxel Feature Interaction for 3D Object Detection

Fully Sparse Fusion for 3D Object Detection