Abstract:Lightweight object detector is currently gaining more and more popularity in remote sensing. In general, it is hard for lightweight detectors to achieve competitive performance compared with the traditional deep models, while knowledge distillation (KD) is a promising training method to tackle the issue. Since the background is more complicated and the object size varies extremely in remote sensing images, it will deliver lots of noise and affect the training performance when directly applying the existing KD methods. To tackle the above problems, we propose an adaptive reinforcement supervision distillation (ARSD) framework to promote the detection capability of the lightweight model. First, we put forward a multiscale core features imitation (MCFI) module for transferring the knowledge of features, which can adaptively select the multiscale core features of objects for distillation and focus more on the features of small objects by an area-weighted strategy. In addition, a strict supervision regression distillation (SSRD) module is designed to select the optimal regression results for distillation, which facilitates the student to effectively imitate the more precise regression output of the teacher network. Massive experiments on a large-scale Dataset for Object deTection in Aerial images (DOTA), object DetectIon in Optical Remote sensing images (DIOR), and Northwestern Polytechnical University Very-High-Resolution 10-class (NWPU VHR) datasets prove that ARSD outperforms the existing distillation state-of-the-art (SOTA) methods. Moreover, the performance of the lightweight model trained with our method transcends other classic heavy and lightweight detectors, which beneficiates the development of lightweight models.

Shoe Print Image Retrieval Based on Dual Knowledge Distillation for Public Security Internet of Things

Research on Knowledge Distillation Algorithm of Object Detection

Self-Paced Knowledge Distillation for Real-Time Image Guided Depth Completion

Learning Slimming SSD Through Pruning and Knowledge Distillation

Pixel Distillation: A New Knowledge Distillation Scheme for Low-Resolution Image Recognition

Feature-Based Knowledge Distillation for Infrared Small Target Detection

Pair-Wise Similarity Knowledge Distillation for RSI Scene Classification

Instance-Aware Distillation for Efficient Object Detection in Remote Sensing Images

Distilling Object Detectors with Global Knowledge

Adaptive Knowledge Distillation for Lightweight Remote Sensing Object Detectors Optimizing

Spirit Distillation: Precise Real-time Semantic Segmentation of Road Scenes with Insufficient Data

DMKD: Improving Feature-based Knowledge Distillation for Object Detection Via Dual Masking Augmentation

DSP-KD: Dual-Stage Progressive Knowledge Distillation for Skin Disease Classification

Two-Way Assistant: A Knowledge Distillation Object Detection Method for Remote Sensing Images

Two-in-one Knowledge Distillation for Efficient Facial Forgery Detection

Knowledge Distillation for Object Detection via Rank Mimicking and Prediction-Guided Feature Imitation

SAKD: Sparse attention knowledge distillation

Self-distillation object segmentation via pyramid knowledge representation and transfer

IDa-Det: An Information Discrepancy-aware Distillation for 1-bit Detectors

Foreground separation knowledge distillation for object detection

Pixel Distillation: Cost-flexible Distillation Across Image Sizes and Heterogeneous Networks