Abstract:Remote sensing image (RSI) scene classification aims to identify semantic categories in RSI using neural networks. However, high-performance deep neural networks typically demand substantial storage and computational resources, making practical deployment challenging. Knowledge distillation has emerged as an effective technique for developing compact models that maintain high classification accuracy in RSI tasks. Existing knowledge distillation methods often overlook the high inter-class similarity in RSI scenes, leading to low-confidence soft labels from the teacher model, which can mislead the student model. Conversely, overly confident soft labels may discard valuable non-target information. Additionally, the significant intra-class variability in RSI contributes to instability in the model's decision boundaries. To address these challenges, we propose an efficient method called instance-level scaling and dynamic margin-alignment knowledge distillation (ISDM) for RSI scene classification. To balance the target and non-target class influence, we apply an entropy regularization loss to scale the teacher model's target class at the instance level. Moreover, we introduce dynamic margin alignment between the student and teacher models to improve the student's discriminative capability. By optimizing soft labels and enhancing the student's ability to distinguish between classes, our method reduces the effects of inter-class similarity and intra-class variability. Experimental results on three public RSI scene classification datasets (AID, UCMerced, and NWPU-RESISC) demonstrate that our method achieves state-of-the-art performance across all teacher–student pairs with lower computational costs. Additionally, we validate the generalization of our approach on general datasets, including CIFAR-100 and ImageNet-1k.

Lightweight Remote Sensing Scene Classification Based on Knowledge Distillation

Lightweight High-Resolution Remote Sensing Scene Classification via Adaptive Enhanced Knowledge Distillation.

From Coarse to Fine: Knowledge Distillation for Remote Sensing Scene Classification

Research on Knowledge Distillation Algorithm of Object Detection

A lightweight and stochastic depth residual attention network for remote sensing scene classification

Remote Sensing Image Scene Classification via Self-Supervised Learning and Knowledge Distillation

Instance-Level Scaling and Dynamic Margin-Alignment Knowledge Distillation for Remote Sensing Image Scene Classification

Pair-Wise Similarity Knowledge Distillation for RSI Scene Classification

Remote Sensing Image Scene Classification with Noisy Label Distillation

Remote Sensing Scene Classification Using Sparse Representation-Based Framework with Deep Feature Fusion

TAKD: Target-Aware Knowledge Distillation for Remote Sensing Scene Classification

A Lightweight Convolution Network with Self-Knowledge Distillation for Hyperspectral Image Classification

Multispectral Scene Classification via Cross-Modal Knowledge Distillation

Fusing Deep Local And Global Features For Remote Sensing Image Scene Classification

RS-SSKD: Self-Supervision Equipped with Knowledge Distillation for Few-Shot Remote Sensing Scene Classification

An Efficient and Lightweight Convolutional Neural Network for Remote Sensing Image Scene Classification

A Remote-Sensing Scene-Image Classification Method Based on Deep Multiple-Instance Learning with a Residual Dense Attention ConvNet

A Lightweight and Discriminative Model for Remote Sensing Scene Classification with Multidilation Pooling Module

Fine-grained Image Classification Based on Knowledge Distillation.

Scene Classification of High-Resolution Remote Sensing Image Using Transfer Learning with Multi-model Feature Extraction Framework.

Adaptive Knowledge Distillation for Lightweight Remote Sensing Object Detectors Optimizing