Abstract:Although detection with Transformer (DETR) is increasingly popular, its global attention modeling requires an extremely long training period to optimize and achieve promising detection performance. Alternative to existing studies that mainly develop advanced feature or embedding designs to tackle the training issue, we point out that the Region-of-Interest (RoI) based detection refinement can easily help mitigate the difficulty of training for DETR methods. Based on this, we introduce a novel REcurrent Glimpse-based decOder (REGO) in this paper. In particular, the REGO employs a multi-stage recurrent processing structure to help the attention of DETR gradually focus on foreground objects more accurately. In each processing stage, visual features are extracted as glimpse features from RoIs with enlarged bounding box areas of detection results from the previous stage. Then, a glimpse-based decoder is introduced to provide refined detection results based on both the glimpse features and the attention modeling outputs of the previous stage. In practice, REGO can be easily embedded in representative DETR variants while maintaining their fully end-to-end training and inference pipelines. In particular, REGO helps Deformable DETR achieve 44.8 AP on the MSCOCO dataset with only 36 training epochs, compared with the first DETR and the Deformable DETR that require 500 and 50 epochs to achieve comparable performance, respectively. Experiments also show that REGO consistently boosts the performance of different DETR detectors by up to 7% relative gain at the same setting of 50 training epochs. Code is available via <a class="link-external link-https" href="https://github.com/zhechen/Deformable-DETR-REGO" rel="external noopener nofollow">this https URL</a>.

GPTR: Gestalt-Perception Transformer for Diagram Object Detection

A Transformer-Based Object Detector with Coarse-Fine Crossing Representations

Diagram Visual Grounding: Learning to See with Gestalt-Perceptual Attention.

Graph Reasoning Transformer for Image Parsing

Bridging the Performance Gap between DETR and R-CNN for Graphical Object Detection in Document Images

Reach the Remote Neighbors: Dual-Encoding Transformer for Graphs

FP-DETR: Detection Transformer Advanced by Fully Pre-training

EGTR: Extracting Graph from Transformer for Scene Graph Generation

GMTR: Graph Matching Transformers

PnP-DETR: Towards Efficient Visual Analysis with Transformers

Recurrent Glimpse-based Decoder for Detection with Transformer

SpaceGTN: A Time-Agnostic Graph Transformer Network for Handwritten Diagram Recognition and Segmentation

QETR: A Query-Enhanced Transformer for Remote Sensing Image Object Detection

Relation Transformer Network

RelTR: Relation Transformer for Scene Graph Generation

Efficient Decoder-Free Object Detection with Transformers

A Bio-Inspired Visual Perception Transformer for Cross-Domain Semantic Segmentation of High-Resolution Remote Sensing Images

Spatial-Temporal Graph Enhanced DETR Towards Multi-Frame 3D Object Detection

MV-DETR: Multi-modality indoor object detection by Multi-View DEtecton TRansformers

Graph Propagation Transformer for Graph Representation Learning

DOCTR: Disentangled Object-Centric Transformer for Point Scene Understanding