Abstract:Visual localization, an integral component of a vast array of computer applications, has been effectively resolved by scene coordinate regression (SCoRe) methods. However, due to the limited receptive field of convolutional neural networks (CNNs), current SCoRe methods have difficulty in distinguishing comparable image patches in sparse texture scenes, thus impairing localization performance. Recently, Transformer exhibits remarkable capability in modelling long-range dependencies, which provides a remedy to the aforementioned problem. Whereas the Transformer alleviates the deficiencies of CNNs, the quadratically computational cost of Transformer leaves it incapable of handling intensive regression tasks, such as scene coordinates prediction. Towards this end, we introduce SACNet, a sparse attention-based network for efficient and accurate visual localization. We overhaul the core designs of vanilla Transformer and further propose a multiple scattered Transformer (MST) with linear complexity. MST consists of a multiple scattered attention (MSA) layer and a filtered feed-forward network (F-FFN). The MSA layer calculates the attention matrix along the channel dimension and adaptively retains the most profitable attention values for feature consolidation such that the consolidated features can better foster scene coordinate regression. F-FFN utilizes a gate mechanism that suppresses less pertinent features, where multi-scale depth-wise convolutions are further used to promote the information flow. After MST, SACNet develops a feature compensator (FC) that combines local geometry features with global context information to predict element-wise soft attention mask, thus enabling the network to adaptively reconcile the importance of local and global-aware local features. Extensive experimental results demonstrate that SACNet noticeably surpasses the cutting-edge methods on several datasets.

Spatially-Aware Context Neural Networks.

SAC-Net: Spatial Attenuation Context for Salient Object Detection

Weakly-Supervised Spatial Context Networks.

DCANet: Dense Context-Aware Network for Semantic Segmentation

SACNet: A Scattered Attention-based Network with Feature Compensator for Visual Localization

Learning to Predict Context-adaptive Convolution for Semantic Segmentation

HCNet: Hierarchical Context Network for Semantic Segmentation

DCNAS: Densely Connected Neural Architecture Search for Semantic Image Segmentation

SCF-Net: Learning Spatial Contextual Features for Large-Scale Point Cloud Segmentation.

Context Spatial Awareness Remote Sensing Image Change Detection Network Based on Graph and Convolution Interaction

Contextual Convolutional Networks

Neuron-level Selective Context Aggregation for Scene Segmentation

OCNet: Object Context Network for Scene Parsing

Attention-guided chained context aggregation for semantic segmentation

Context Encoding for Semantic Segmentation

Spatial-Context-Aware Deep Neural Network for Multi-Class Image Classification

Long and short-range relevance context network for semantic segmentation

Learning discriminative context for salient object detection

ContextNet: Exploring Context and Detail for Semantic Segmentation in Real-time

Global Context Encoding For Salient Objects Detection

Deep Salient Object Detection with Contextual Information Guidance