Abstract:Recently, DETR-like detectors, which have shown remarkable performance in general object detection, face limitations when dealing with remote sensing images primarily containing small objects. Mainstream two-stage DETR-like models employ a pipeline that selects and processes a small portion of informative tokens, which enhances performance but also shows a high dependency on token selection. The current static token selection strategies lead to inconsistencies between the static selection criteria and dynamic token updates. Additionally, in remote sensing images, the limited information available for small objects and their inherent sensitivity to pixel shifts further degrade detection performance. To address this, we propose Scale-Adaptive Salience DETR (SAS DETR), a two-stage DETR-like method. SAS DETR incorporates dynamic token filtering, which uses a global threshold predictor to determine the token filtering ratio for each layer of the encoder. This approach selects an appropriate filtering ratio for different network layers while maintaining consistency between the foreground confidence map and token updates. Furthermore, we introduce a novel scale-adaptive salience supervision mechanism that adaptively scales the salience computation area based on object size, ensuring the model more effectively supervises small objects and utilizes the information within tokens without compromising the detection performance for objects of other sizes. Finally, we employ Scale-Adaptive Intersection over Union to reduce the impact of pixel shifts on small objects. With these improvements, our proposed SAS DETR achieves 25.2% AP on the AI-TOD-V2 dataset with 24 training epochs and 50.4% AP on the COCO 2017 dataset with 12 training epochs.

SA-DETR: Saliency Attention-based DETR for salient object detection

Salience DETR: Enhancing Detection Transformer with Hierarchical Salience Filtering Refinement

SA-DPNet: Structure-aware dual pyramid network for salient object detection

Decoupled DETR: Spatially Disentangling Localization and Classification for Improved End-to-End Object Detection

Sparse DETR: Efficient End-to-End Object Detection with Learnable Sparsity

DETR with Additional Global Aggregation for Cross-domain Weakly Supervised Object Detection

Transformer with large convolution kernel decoder network for salient object detection in optical remote sensing images

Densely Deformable Efficient Salient Object Detection Network

Semantic feature-guided and correlation-aggregated salient object detection

SAP-DETR: Bridging the Gap Between Salient Points and Queries-Based Transformer Detector for Fast Model Convergency

Scale-Adaptive Salience Supervision and Dynamic Token Filtering for Small Object Detection in Remote Sensing Images

Sparse Semi-DETR: Sparse Learnable Queries for Semi-Supervised Object Detection

CSDNet: Detect Salient Object in Depth-Thermal via A Lightweight Cross Shallow and Deep Perception Network

Pyramidal Attention for Saliency Detection

A Unified Structure for Efficient RGB and RGB-D Salient Object Detection

An Improved DETR Based on Angle Denoising and Oriented Boxes Refinement for Remote Sensing Object Detection

Deformable DETR: Deformable Transformers for End-to-End Object Detection

DeepSaliency: Multi-Task Deep Neural Network Model for Salient Object Detection

DAGCN: Dynamic and Adaptive Graph Convolutional Network for Salient Object Detection.

Richer and Deeper Supervision Network for Salient Object Detection

CLASS: Cross-Level Attention and Supervision for Salient Objects Detection