SCCMDet: Adaptive Sparse Convolutional Networks Based on Class Maps for Real-Time Onboard Detection in Unmanned Aerial Vehicle Remote Sensing Images

Qifan Tan,Xuqi Yang,Cheng Qiu,Yanhuan Jiang,Jinze He,Jingshuo Liu,Yahui Wu

DOI: https://doi.org/10.3390/rs16061031

IF: 5

2024-03-15

Remote Sensing

Abstract:Onboard, real-time object detection in unmaned aerial vehicle remote sensing (UAV-RS) has always been a prominent challenge due to the higher image resolution required and the limited computing resources available. Due to the trade-off between accuracy and efficiency, the advantages of UAV-RS are difficult to fully exploit. Current sparse-convolution-based detectors only convolve some of the meaningful features in order to accelerate the inference speed. However, the best approach to the selection of meaningful features, which ultimately determines the performance, is an open question. This study proposes the use of adaptive sparse convolutional networks based on class maps for real-time onboard detection in UAV-RS images (SCCMDet) to solve this problem. For data pre-processing, SCCMDet obtains the real class maps as labels from the ground truth to supervise the feature selection process. In addition, a generate class map network (GCMN), equipped with a newly designed loss function, identifies the importance of features to generate a binary class map which filters the image for its more meaningful sparse features. Comparative experiments were conducted on the VisDrone dataset, and the experimental results show that our method accelerates YOLOv8 by 41.94% at most and increases the performance by 2.52%. Moreover, ablation experiments demonstrate the effectiveness of the proposed model.

environmental sciences,imaging science & photographic technology,remote sensing,geosciences, multidisciplinary

What problem does this paper attempt to address?

The paper primarily addresses the issue of real-time onboard target detection in UAV remote sensing (UAV-RS) images. Due to the requirement for high-resolution images and limited computational resources, achieving efficient and accurate target detection has always been a challenge in the field of UAV remote sensing. Although existing sparse convolution methods can accelerate the inference process, how to select meaningful features remains an open question.

To address the above issues, the authors propose SCCMDet (Class Map-based Adaptive Sparse Convolution Network), a method for real-time onboard detection in UAV remote sensing images. The core contributions of SCCMDet include:

1. **Proposing the SCCMDet model**: By generating "real class maps" from real annotations to guide feature selection, the convolution process of sparse features is accelerated. This method improves the speed of real-time processing of high-resolution images, utilizing limited onboard computational resources.

2. **Designing the Generative Class Map Network (GCMN)**: To obtain reasonable class maps, the paper introduces the GCMN structure and designs a new loss function to evaluate the quality of the class maps generated by GCMN, thereby providing feedback to the network for further optimization.

3. **Experimental validation**: Experimental results on the VisDrone dataset show that SCCMDet can significantly reduce computational costs, accelerating up to 41.94% compared to the YOLOv8 model, with a performance improvement of 2.52%.

In summary, this paper aims to improve the efficiency and accuracy of target detection in UAV remote sensing images by proposing a novel method, particularly achieving real-time processing capabilities under limited computational resources.