Abstract:Building extraction is a challenging research direction in remote sensing image (RSI) interpretation. Due to the fact that a building has not only its own local structures but also similar architectural styles with other buildings located in a global area (e.g., street or community), fusing local and global features becomes a promising way to improve performance of building extraction. Focused on this, we propose a new complementarity-aware local-global feature fusion network (CLGFF-Net) by integrating a convolutional branch and a Transformer branch. The two branches respectively capture local patterns and global long-range dependencies of RSIs, thereby leading to highly complementary features. To dig out the implicit complementary information for fusion, we develop a complementarity-aware fusion module (CFM) which separates shared features (SFs) and distinct features (DFs) between two branches, by building a commonalities analysis path and two difference analysis paths. Meanwhile, to make sure the similarity of SFs and dissimilarity of DFs, a triplet loss function is designed to enforce the distances between SFs to be near and DFs to be far. By this way, complementary information can be explicitly included in DFs and is adaptively exchanged between two branches for fusion. Besides, since multilayer features in each branch generally convey different-level semantic information, a multi-layer fusion scheme (MLFS) is designed to fuse them by introducing cross-layer connections and gate mechanism. By coupling CFMs with MLFS, the abilities in characterizing local and global context information, as well as different-level semantic information, can be fully exploited for better mapping of complicated building objects. Experimental results demonstrate the effectiveness of our proposed method.

InterFormer: Interactive Local and Global Features Fusion for Automatic Speech Recognition

FusionFormer: Fusing Operations in Transformer for Efficient Streaming Speech Recognition

FusionFormer: Fusing Operations in Transformer for Efficient Streaming Speech Recognition

InterFormer: Human Interaction Understanding with Deformed Transformer.

An Enhanced Res2Net with Local and Global Feature Fusion for Speaker Verification

SPformer: Hybrid Sequential-Parallel Architectures for Automatic Speech Recognition

Multi-pass Training and Cross-information Fusion for Low-resource End-to-end Accented Speech Recognition

Audio-Visual Speech Recognition Using A Two-Step Feature Fusion Strategy.

Effective Cross-Utterance Language Modeling for Conversational Speech Recognition

Attentive Fusion Enhanced Audio-Visual Encoding for Transformer Based Robust Speech Recognition

Multi-Scale Hybrid Fusion Network for Mandarin Audio-Visual Speech Recognition

Attention-Guided Multi-scale Interaction Network for Face Super-Resolution

Fusion of deep shallow features and models for speaker recognition

Attentive Feature Fusion for Robust Speaker Verification

Acoustic Model Fusion for End-to-end Speech Recognition

Fusion of Discrete Representations and Self-Augmented Representations for Multilingual Automatic Speech Recognition

Divide, Conquer and Combine: Hierarchical Feature Fusion Network with Local and Global Perspectives for Multimodal Affective Computing.

Robust Audio-Visual Speech Recognition Based on Hybrid Fusion

Improving Transformer-based Conversational ASR by Inter-Sentential Attention Mechanism

GLFuse: A Global and Local Four-Branch Feature Extraction Network for Infrared and Visible Image Fusion

Complementarity-aware Local-global Feature Fusion Network for Building Extraction in Remote Sensing Images