Abstract:Remote sensing image (RSI) scene classification aims to identify semantic categories in RSI using neural networks. However, high-performance deep neural networks typically demand substantial storage and computational resources, making practical deployment challenging. Knowledge distillation has emerged as an effective technique for developing compact models that maintain high classification accuracy in RSI tasks. Existing knowledge distillation methods often overlook the high inter-class similarity in RSI scenes, leading to low-confidence soft labels from the teacher model, which can mislead the student model. Conversely, overly confident soft labels may discard valuable non-target information. Additionally, the significant intra-class variability in RSI contributes to instability in the model's decision boundaries. To address these challenges, we propose an efficient method called instance-level scaling and dynamic margin-alignment knowledge distillation (ISDM) for RSI scene classification. To balance the target and non-target class influence, we apply an entropy regularization loss to scale the teacher model's target class at the instance level. Moreover, we introduce dynamic margin alignment between the student and teacher models to improve the student's discriminative capability. By optimizing soft labels and enhancing the student's ability to distinguish between classes, our method reduces the effects of inter-class similarity and intra-class variability. Experimental results on three public RSI scene classification datasets (AID, UCMerced, and NWPU-RESISC) demonstrate that our method achieves state-of-the-art performance across all teacher–student pairs with lower computational costs. Additionally, we validate the generalization of our approach on general datasets, including CIFAR-100 and ImageNet-1k.

TAKD: Target-Aware Knowledge Distillation for Remote Sensing Scene Classification

Research on Knowledge Distillation Algorithm of Object Detection

DCCD: Reducing Neural Network Redundancy Via Distillation

Target Detection Model Distillation Using Feature Transition and Label Registration for Remote Sensing Imagery

Instance-Level Scaling and Dynamic Margin-Alignment Knowledge Distillation for Remote Sensing Image Scene Classification

Consistency- and dependence-guided knowledge distillation for object detection in remote sensing images

Two-Way Assistant: A Knowledge Distillation Object Detection Method for Remote Sensing Images

Pair-Wise Similarity Knowledge Distillation for RSI Scene Classification

Adaptive Knowledge Distillation for Lightweight Remote Sensing Object Detectors Optimizing

ERKT-Net: Implementing Efficient and Robust Knowledge Distillation for Remote Sensing Image Classification

Lightweight Remote Sensing Scene Classification Based on Knowledge Distillation

Feature-Based Knowledge Distillation for Infrared Small Target Detection

Efficient knowledge distillation for hybrid models: A vision transformer‐convolutional neural network to convolutional neural network approach for classifying remote sensing images

Instance-Aware Distillation for Efficient Object Detection in Remote Sensing Images

SAKD: Sparse attention knowledge distillation

Pixel Distillation: A New Knowledge Distillation Scheme for Low-Resolution Image Recognition

Teacher-student collaborative knowledge distillation for image classification

Knowledge Distillation for Oriented Object Detection on Aerial Images

RDR-KD: A Knowledge Distillation Detection Framework for Drone Scenes

Structured Knowledge Distillation for Accurate and Efficient Object Detection

Efficient Object Detection in Optical Remote Sensing Imagery via Attention-Based Feature Distillation