Abstract:Minimizing the computation complexity is essential for the popularization of deep networks in practical applications. Nowadays, most researches attempt to accelerate deep networks by designing new network structure or compressing the network parameters. Meanwhile, transfer learning techniques such as knowledge distillation are utilized to keep the performance of deep models. In this paper, we focus on accelerating deep models and relieving the computation burden by using low-resolution (LR) images as inputs while maintaining competitive performance, which is rarely researched in the current literature. Deep networks may encounter serious performance degradation when using LR inputs because many details are unavailable from LR images. Besides, the existing approaches may fail to learn discriminative features for LR images because of the dramatic appearance variations between LR and high-resolution (HR) images. To tackle with the above problems, we propose a resolution-aware knowledge distillation (RKD) framework to narrow the cross-resolution variations by transferring knowledge from HR domain to LR domain. The proposed framework consists of a HR teacher network and a LR student network. First, we introduce a discriminator and propose an adversarial learning strategy to shrink the variations between inputs with changing resolution. Then we design a cross-resolution knowledge distillation (CRKD) loss to train discriminative student network by exploiting the knowledge of the teacher network. The CRKD loss is consisted of a resolution-aware distillation loss, a pair-wise constraint, and a maximum mean discrepancy loss. Experimental results on person re-identification, image classification, face recognition, and defect segmentation tasks demonstrate that RKD outperforms traditional knowledge distillation method by achieving better performance with lower computation complexities. Furthermore, CRKD surpasses the state-of-the-art knowledge distillation methods in transferring knowledge across different resolutions under RKD framework, especially when coping with large resolution differences.

Viewpoint Robust Knowledge Distillation for Accelerating Vehicle Re-identification

Efficient but Lightweight Network for Vehicle Re-Identification with Center-Constraint Loss

Viewpoint Adaptation Learning with Cross-View Distance Metric for Robust Vehicle Re-Identification.

VARID: Viewpoint-Aware Re-IDentification of Vehicle Based on Triplet Loss

Multi‐label Based View Learning for Vehicle Re‐identification

Viewpoint-aware Progressive Clustering for Unsupervised Vehicle Re-identification

Viewpoint Alignment and Discriminative Parts Enhancement in 3D Space for Vehicle ReID

Pluggable Weakly-Supervised Cross-View Learning for Accurate Vehicle Re-Identification

Learning More in Vehicle Re-Identification: Joint Local Blur Transformation and Adversarial Network Optimization

Discriminative-region attention and orthogonal-view generation model for vehicle re-identification

Knowledge self-distillation for visible-infrared cross-modality person re-identification

Attributes Guided Feature Learning for Vehicle Re-Identification

Efficient and Deep Vehicle Re-Identification Using Multi-Level Feature Extraction

Multi-View Spatial Attention Embedding for Vehicle Re-Identification

Vehicle Re-Identification Using Quadruple Directional Deep Learning Features

Viewpoint and Scale Consistency Reinforcement for UAV Vehicle Re-Identification.

Vehicle Re-Identification Using Distance-Based Global and Partial Multi-Regional Feature Learning

V2ReID: Vision-Outlooker-Based Vehicle Re-Identification

MULTI-ATTRIBUTE DRIVEN VEHICLE RE-IDENTIFICATION WITH SPATIAL-TEMPORAL RE-RANKING

VOC-ReID: Vehicle Re-identification based on Vehicle-Orientation-Camera

Resolution-Aware Knowledge Distillation for Efficient Inference.