Abstract:Transformers have revolutionized the object detection landscape by introducing DETRs, acclaimed for their simplicity and efficacy. Despite their advantages, the substantial size of these models poses significant challenges for practical deployment, particularly in resource-constrained environments. This paper addresses the challenge of compressing DETR by leveraging knowledge distillation, a technique that holds promise for maintaining model performance while reducing size. A critical aspect of DETRs' performance is their reliance on queries to interpret object representations accurately. Traditional distillation methods often focus exclusively on positive queries, identified through bipartite matching, neglecting the rich information present in hard-negative queries. Our visual analysis indicates that hard-negative queries, focusing on foreground elements, are crucial for enhancing distillation outcomes. To this end, we introduce a novel Group Query Selection strategy, which diverges from traditional query selection in DETR distillation by segmenting queries based on their Generalized Intersection over Union (GIoU) with ground truth objects, thereby uncovering valuable hard-negative queries for distillation. Furthermore, we present the Knowledge Distillation via Query Selection for DETR (QSKD) framework, which incorporates Attention-Guided Feature Distillation (AGFD) and Local Alignment Prediction Distillation (LAPD). These components optimize the distillation process by focusing on the most informative aspects of the teacher model's intermediate features and output. Our comprehensive experimental evaluation of the MS-COCO dataset demonstrates the effectiveness of our approach, significantly improving average precision (AP) across various DETR architectures without incurring substantial computational costs. Specifically, the AP of Conditional DETR ResNet-18 increased from 35.8 to 39.9.

Distillation Using Oracle Queries for Transformer-based Human-Object Interaction Detection

Human–object interaction detection based on disentangled axial attention transformer

Parallel disentangling network for human–object interaction detection

Human-Object Interaction Detection via Disentangled Transformer

End-to-End Human Object Interaction Detection with HOI Transformer

Category-Aware Transformer Network for Better Human-Object Interaction Detection

Pairwise CNN-Transformer Features for Human–Object Interaction Detection

Disentangled Pre-training for Human-Object Interaction Detection

Knowledge Distillation via Query Selection for Detection Transformer

Point-Based Learnable Query Generator for Human–Object Interaction Detection

Orchestrating the Symphony of Prompt Distribution Learning for Human-Object Interaction Detection

A Novel Part Refinement Tandem Transformer for Human-Object Interaction Detection

CL-HOI: Cross-Level Human-Object Interaction Distillation from Vision Large Language Models

Towards Hard-Positive Query Mining for DETR-Based Human-Object Interaction Detection.

Disentangled Interaction Representation for One-Stage Human-Object Interaction Detection

UAHOI: Uncertainty-aware Robust Interaction Learning for HOI Detection

HODN: Disentangling Human-Object Feature for HOI Detection

Exploring Structure-aware Transformer over Interaction Proposals for Human-Object Interaction Detection

HOD: Human-Object Decoupling Network for HOI Detection

Mask-Guided Transformer for Human-Object Interaction Detection