Abstract:Scene classification is a key step in intelligent information processing of high-resolution remote sensing, which aims to identify the land-use types of remote sensing image blocks. In recent years, the models based on deep convolution neural networks (CNN) have made outstanding achievements in the field of remote sensing, but these models are computationally expensive and time-consuming, while the lightweight shallow network models have few parameters, fast speed, but low accuracy. Both the deep CNN models and the lightweight shallow network models could not be directly applied to embedded devices. Therefore, we propose an Adaptive Enhanced Knowledge Distillation (AE-KD) to deeply mine the output and feature information from teacher model and transfer them to student model, so as to improve the performance of lightweight model. Firstly, aiming at the uneven degree of difference among remote sensing image categories, an adaptive temperature mechanism is proposed by improving the temperature mechanism in the traditional knowledge distillation, which promotes the student model to better learn the probability distribution knowledge from the output layer of the large and deep teacher model. And then, the spatial attention and inter-channel correlation of features are added as constraints in order to make the student model learn the multi-level knowledge from teacher model. The experimental results on UC Merced Land-Use and AID public datasets show that the proposed method reduces 91 % parameters of the teacher model and improves the prediction speed by 22 times, where it has only a small loss of classification accuracy, which is effective for the lightweight model. ablation study also further analyzes and discusses the performance improvement of the student model under different levels of knowledge distillation.

A Knowledge Distillation Method Based on IQE Attention Mechanism for Target Recognition in Sar Imagery.

Research on Knowledge Distillation Algorithm of Object Detection

An IQE Criterion-Based Method for SAR Images Classification Network Pruning.

TAKD: Target-Aware Knowledge Distillation for Remote Sensing Scene Classification

Knowledge transfer evolutionary search for lightweight neural architecture with dynamic inference

EdgeKE: An On-Demand Deep Learning IoT System for Cognitive Big Data on Industrial Edge Devices

Boosting Lightweight CNNs Through Network Pruning and Knowledge Distillation for SAR Target Recognition.

Feature-Based Knowledge Distillation for Infrared Small Target Detection

Cooperative Knowledge-Distillation-Based Tiny DNN for UAV-Assisted Mobile-Edge Network

SAR Target Classification Based on Knowledge Distillation

Localized Knowledge Distillation Helps IoT Devices Provide High-performance Visual Services

Lightweight High-Resolution Remote Sensing Scene Classification via Adaptive Enhanced Knowledge Distillation.

Pair-Wise Similarity Knowledge Distillation for RSI Scene Classification

Pixel Distillation: A New Knowledge Distillation Scheme for Low-Resolution Image Recognition

ERKT-Net: Implementing Efficient and Robust Knowledge Distillation for Remote Sensing Image Classification

EKDF: an Ensemble Knowledge Distillation Framework for Robust Collaborative Inference on Heterogeneous Edge Devices

An Efficient Knowledge Distillation-Based Detection Method for Infrared Small Targets

Knowledge Distillation-Based Lightweight Change Detection in High-Resolution Remote Sensing Imagery for On-Board Processing

A Lightweight SAR Image Recognition Algorithm Based on Deep Convolutional Neural Network

Fine-grained Image Classification Based on Knowledge Distillation.

Enabling Deep Learning on Edge Devices through Filter Pruning and Knowledge Transfer