Abstract:Object detection is a fundamental task in remote sensing image analysis and scene understanding. Previous remote sensing object detectors are typically based on convolutional neural networks (CNNs), whose performance is significantly limited by the intrinsic locality of convolution operations. The emergence of vision Transformers brings potential solutions to this problem, which has the capability to be a solid alternative to CNNs. However, three crucial obstacles hinder the application and performance of Transformers in the task of remote sensing object detection, that is: 1) high computational complexity, especially for high-resolution remote sensing images; 2) training and sample inefficiency caused by lack of inductive bias; and 3) difficulty in learning arbitrary orientation knowledge of geospatial objects. To address these issues, in this article, a novel efficient inductive vision Transformer framework is proposed for oriented object detection in remote sensing imagery. This framework follows the hierarchical feature pyramid structure and makes threefold contributions as follows: 1) spatial redundancy in remote sensing images is fully explored and an adaptive multigrained routing mechanism is proposed to facilitate token sparsity in Transformers, which can dramatically reduce the computational cost without comprising the accuracy. 2) A compact dual-path encoding architecture, where both global long-range dependencies and local semantic relations are jointly and complementarily captured, is proposed to enhance inductive bias in Transformers. 3) An angle tokenization technique is proposed to promote the encoding, embedding, and learning of direction knowledge for oriented objects in remote sensing scenarios. In this work, the above-mentioned three contributions are instantiated in an advanced Transformer-based object detector, namely, EIA-pyramid vision Transformer (PVT). Comprehensive experiments on two publicly available datasets have demonstrated its effectiveness and superiority for oriented object detection in remote sensing images.

Wavelet Tree Transformer: Multihead Attention With Frequency-Selective Representation and Interaction for Remote Sensing Object Detection

Wavelet Tree Transformer: Multi-Head Attention with Frequency Selective Representation and Interaction for Remote Sensing Object Detection

Hyperspectral Image Classification Using Group-Aware Hierarchical Transformer

Hierarchical Feature Fusion of Transformer with Patch Dilating for Remote Sensing Scene Classification

Wave-ViT: Unifying Wavelet and Transformers for Visual Representation Learning

DSViT: Dynamically Scalable Vision Transformer for Remote Sensing Image Segmentation and Classification.

Deep Hierarchical Vision Transformer for Hyperspectral and LiDAR Data Classification

Efficient Inductive Vision Transformer for Oriented Object Detection in Remote Sensing Imagery

Enhanced Window-Based Self-Attention with Global and Multi-Scale Representations for Remote Sensing Image Super-Resolution

ASAFormer: Visual tracking with convolutional vision transformer and asymmetric selective attention

HAM-Transformer: A Hybrid Adaptive Multi-Scaled Transformer Net for Remote Sensing in Complex Scenes

Deep Spectral Spatial Feature Enhancement Through Transformer for Hyperspectral Image Classification

Attention Head Interactive Dual Attention Transformer for Hyperspectral Image Classification

Multiattention Joint Convolution Feature Representation with Lightweight Transformer for Hyperspectral Image Classification.

CAW: A Remote-Sensing Scene Classification Network Aided by Local Window Attention

Deep hierarchical transformer for change detection in high-resolution remote sensing images

Advancing Plain Vision Transformer Towards Remote Sensing Foundation Model

HA-Transformer: Harmonious aggregation from local to global for object detection

Triple Attention Vision Transformers for Remote Sensing Image Classification

Investigation of Hierarchical Spectral Vision Transformer Architecture for Classification of Hyperspectral Imagery

VisionTwinNet: Gated Clarity Enhancement Paired With Light-Robust CD Transformers