Abstract:Object detection in aerial images has always been a challenging task due to the generally small size of the objects. Most current detectors prioritize novel detection frameworks, often overlooking research on fundamental components such as feature pyramid networks. In this paper, we introduce the Cross-Layer Feature Pyramid Transformer (CFPT), a novel upsampler-free feature pyramid network designed specifically for small object detection in aerial images. CFPT incorporates two meticulously designed attention blocks with linear computational complexity: the Cross-Layer Channel-Wise Attention (CCA) and the Cross-Layer Spatial-Wise Attention (CSA). CCA achieves cross-layer interaction by dividing channel-wise token groups to perceive cross-layer global information along the spatial dimension, while CSA completes cross-layer interaction by dividing spatial-wise token groups to perceive cross-layer global information along the channel dimension. By integrating these modules, CFPT enables cross-layer interaction in one step, thereby avoiding the semantic gap and information loss associated with element-wise summation and layer-by-layer transmission. Furthermore, CFPT incorporates global contextual information, which enhances detection performance for small objects. To further enhance location awareness during cross-layer interaction, we propose the Cross-Layer Consistent Relative Positional Encoding (CCPE) based on inter-layer mutual receptive fields. We evaluate the effectiveness of CFPT on two challenging object detection datasets in aerial images, namely VisDrone2019-DET and TinyPerson. Extensive experiments demonstrate the effectiveness of CFPT, which outperforms state-of-the-art feature pyramid networks while incurring lower computational costs. The code will be released at <a class="link-external link-https" href="https://github.com/duzw9311/CFPT" rel="external noopener nofollow">this https URL</a>.

Transformer-Based Context Condensation for Boosting Feature Pyramids in Object Detection

A Transformer-Based Object Detector with Coarse-Fine Crossing Representations

Feature Shrinkage Pyramid for Camouflaged Object Detection with Transformers.

Multi-scale Global Context Feature Pyramid Network for Object Detector

Context-Transformer: Tackling Object Confusion for Few-Shot Detection

Transformed Dynamic Feature Pyramid for Small Object Detection

Tiny object detection with context enhancement and feature purification

CB-FPN: object detection feature pyramid network based on context information and bidirectional efficient fusion

Multi-scale Context Enhancement Network for Object Detection

Feature Combination Based On Receptive Fields And Cross-Fusion Feature Pyramid For Object Detection

Object Detector with Enriched Global Context Information

Complementary Feature Pyramid Network for Object Detection

CFPFormer: Feature-pyramid like Transformer Decoder for Segmentation and Detection

Bi-directional Skip Connection Feature Pyramid Network and Sub-Pixel Convolution for High-Quality Object Detection

Integrally Migrating Pre-trained Transformer Encoder-decoders for Visual Object Detection

Adaptive Pyramid Context Fusion for Point Cloud Perception

Multi-scale Fusion with Context-aware Network for Object Detection

DETR++: Taming Your Multi-Scale Detection Transformer

Cross-Layer Feature Pyramid Transformer for Small Object Detection in Aerial Images

FPNFormer: Rethink the Method of Processing the Rotation-Invariance and Rotation-Equivariance on Arbitrary-Oriented Object Detection

Combining transformer global and local feature extraction for object detection