Abstract:In this paper, we present all-embracing Transformers (AaTs) that are capable of deftly manipulating attention mechanism for Received Signal Strength (RSS) fingerprints in order to invigorate localizing performance. Since most machine learning models applied to the RSS modality do not possess any attention mechanism, they can merely capture superficial representations. Moreover, compared to textual and visual modalities, the RSS modality is inherently notorious for its sensitivity to environmental dynamics. Such adversities inhibit their access to subtle but distinct representations that characterize the corresponding location, ultimately resulting in significant degradation in the testing phase. In contrast, a major appeal of AaTs is the ability to focus exclusively on relevant anchors in RSS sequences, allowing full rein to the exploitation of subtle and distinct representations for specific locations. This also facilitates disregarding redundant clues formed by noisy ambient conditions, thus enhancing accuracy in localization. Apart from that, explicitly resolving the representation collapse (i.e., none-informative or homogeneous features, and gradient vanishing) can further invigorate the self-attention process in transformer blocks, by which subtle but distinct representations to specific locations are radically captured with ease. For that purpose, we first enhance our proposed model with two sub-constraints, namely covariance and variance losses at the Anchor2Vec. The proposed constraints are automatically mediated with the primary task towards a novel multi-task learning manner. In an advanced manner, we present further the ultimate in design with a few simple tweaks carefully crafted for transformer encoder blocks. This effort aims to promote representation augmentation via stabilizing the inflow of gradients to these blocks. Thus, the problems of representation collapse in regular Transformers can be tackled. To evaluate our AaTs, we compare the models with the state-of-the-art (SoTA) methods on three benchmark indoor localization datasets. The experimental results confirm our hypothesis and show that our proposed models could deliver much higher and more stable accuracy.

AAFormer: Attention-Attended Transformer for Semantic Segmentation of Remote Sensing Images

Rethinking Transformers for Semantic Segmentation of Remote Sensing Images.

Locality-Enhanced Transformer for Semantic Segmentation of High-Resolution Remote Sensing Images.

AerialFormer: Multi-resolution Transformer for Aerial Image Segmentation

ACTNet: A Dual-Attention Adapter with a CNN-Transformer Network for the Semantic Segmentation of Remote Sensing Imagery

Transformer Meets Convolution: A Bilateral Awareness Network for Semantic Segmentation of Very Fine Resolution Urban Scene Images

Hybrid Attention Fusion Embedded in Transformer for Remote Sensing Image Semantic Segmentation

Semantic segmentation using cross-stage feature reweighting and efficient self-attention

Low-Resolution Self-Attention for Semantic Segmentation

Enhanced Window-Based Self-Attention with Global and Multi-Scale Representations for Remote Sensing Image Super-Resolution

ASAFormer: Visual tracking with convolutional vision transformer and asymmetric selective attention

AVESFormer: Efficient Transformer Design for Real-Time Audio-Visual Segmentation

Semantic Segmentation Based on Vision Transformer Via Interactive Attention

When zero-padding position encoding encounters linear space reduction attention: an efficient semantic segmentation Transformer of remote sensing images

A transformer-based approach empowered by a self-attention technique for semantic segmentation in remote sensing

AANet: Adaptive Attention Networks for Semantic Segmentation of High-Resolution Remote Sensing Imagery

LBFormer: Scene Perception Segmentation Transformer Based on Local Block

RCCT-ASPPNet: Dual-Encoder Remote Image Segmentation Based on Transformer and ASPP

Seeing the world from its words: All-embracing Transformers for fingerprint-based indoor localization

Efficient Transformer for Remote Sensing Image Segmentation

XANet: An Efficient Remote Sensing Image Segmentation Model Using Element-Wise Attention Enhancement and Multi-Scale Attention Fusion