Abstract:Visual localization, an integral component of a vast array of computer applications, has been effectively resolved by scene coordinate regression (SCoRe) methods. However, due to the limited receptive field of convolutional neural networks (CNNs), current SCoRe methods have difficulty in distinguishing comparable image patches in sparse texture scenes, thus impairing localization performance. Recently, Transformer exhibits remarkable capability in modelling long-range dependencies, which provides a remedy to the aforementioned problem. Whereas the Transformer alleviates the deficiencies of CNNs, the quadratically computational cost of Transformer leaves it incapable of handling intensive regression tasks, such as scene coordinates prediction. Towards this end, we introduce SACNet, a sparse attention-based network for efficient and accurate visual localization. We overhaul the core designs of vanilla Transformer and further propose a multiple scattered Transformer (MST) with linear complexity. MST consists of a multiple scattered attention (MSA) layer and a filtered feed-forward network (F-FFN). The MSA layer calculates the attention matrix along the channel dimension and adaptively retains the most profitable attention values for feature consolidation such that the consolidated features can better foster scene coordinate regression. F-FFN utilizes a gate mechanism that suppresses less pertinent features, where multi-scale depth-wise convolutions are further used to promote the information flow. After MST, SACNet develops a feature compensator (FC) that combines local geometry features with global context information to predict element-wise soft attention mask, thus enabling the network to adaptively reconcile the importance of local and global-aware local features. Extensive experimental results demonstrate that SACNet noticeably surpasses the cutting-edge methods on several datasets.

TCNet: tensor and covariance attention network for semantic segmentation

Deep Dual-Stream Network with Scale Context Selection Attention Module for Semantic Segmentation

TCNet: Multiscale Fusion of Transformer and CNN for Semantic Segmentation of Remote Sensing Images

Attention-guided chained context aggregation for semantic segmentation

Threshold Attention Network for Semantic Segmentation of Remote Sensing Images

Attention Guided Global Enhancement and Local Refinement Network for Semantic Segmentation

HCNet: Hierarchical Context Network for Semantic Segmentation

CI-Net: a joint depth estimation and semantic segmentation network using contextual information

A Synergistical Attention Model for Semantic Segmentation of Remote Sensing Images

LANet: Local Attention Embedding to Improve the Semantic Segmentation of Remote Sensing Images.

TA-Net: Triple attention network for medical image segmentation

TCANet: three-stream coordinate attention network for RGB-D indoor semantic segmentation

Learning Cross-Channel Representations for Semantic Segmentation

SSNet: A Novel Transformer and CNN Hybrid Network for Remote Sensing Semantic Segmentation

SACNet: A Scattered Attention-based Network with Feature Compensator for Visual Localization

Class Semantic Enhancement Network for Semantic Segmentation

Point Attention Network for Semantic Segmentation of 3D Point Clouds

SCAttNet: Semantic Segmentation Network with Spatial and Channel Attention Mechanism for High-Resolution Remote Sensing Images

CAA : Channelized Axial Attention for Semantic Segmentation.

SACANet: scene-aware class attention network for semantic segmentation of remote sensing images

Adaptive multi-scale dual attention network for semantic segmentation