Abstract:Remote Sensing Visual Question Answering (RSVQA) has gained significant research interest. However, current RSVQA methods are limited by the imaging mechanisms of optical sensors, particularly under challenging conditions such as cloud-covered and low-light scenarios. Given the all-time and all-weather imaging capabilities of Synthetic Aperture Radar (SAR), it is crucial to investigate the integration of optical-SAR images to improve RSVQA performance. In this work, we propose a Text-guided Coarse-to-Fine Fusion Network (TGFNet), which leverages the semantic relationships between question text and multi-source images to guide the network toward complementary fusion at the feature level. Specifically, we develop a Text-guided Coarse-to-Fine Attention Refinement (CFAR) module to focus on key areas related to the question in complex remote sensing images. This module progressively directs attention from broad areas to finer details through key region routing, enhancing the model's ability to focus on relevant regions. Furthermore, we propose an Adaptive Multi-Expert Fusion (AMEF) module that dynamically integrates different experts, enabling the adaptive fusion of optical and SAR features. In addition, we create the first large-scale benchmark dataset for evaluating optical-SAR RSVQA methods, comprising 6,008 well-aligned optical-SAR image pairs and 1,036,694 well-labeled question-answer pairs across 16 diverse question types, including complex relational reasoning questions. Extensive experiments on the proposed dataset demonstrate that our TGFNet effectively integrates complementary information between optical and SAR images, significantly improving the model's performance in challenging scenarios. The dataset is available at: <a class="link-external link-https" href="https://github.com/mmic-lcl/" rel="external noopener nofollow">this https URL</a>. Index Terms: Remote Sensing Visual Question Answering, Multi-source Data Fusion, Multimodal, Remote Sensing, OPT-SAR.

Injecting Linguistic into Visual Backbone: Query-Aware Multimodal Fusion Network for Remote Sensing Visual Grounding

A Spatial Frequency Fusion Strategy Based on Linguistic Query Refinement for RSVG

Multi-Stage Synergistic Aggregation Network for Remote Sensing Visual Grounding

Text-Guided Coarse-to-Fine Fusion Network for Robust Remote Sensing Visual Question Answering

Language Query-Based Transformer With Multiscale Cross-Modal Alignment for Visual Grounding on Remote Sensing Images

Language Query Based Transformer with Multi-Scale Cross-Modal Alignment for Visual Grounding on Remote Sensing Images

Multi-Attention Fusion and Fine-Grained Alignment for Bidirectional Image-Sentence Retrieval in Remote Sensing

Mutual Attention Inception Network for Remote Sensing Visual Question Answering

Enhancing Remote Sensing Visual Question Answering: A Mask-Based Dual-Stream Feature Mutual Attention Network

Shifting More Attention to Visual Backbone: Query-modulated Refinement Networks for End-to-End Visual Grounding

Attribute-Prompting Multi-Modal Object Reasoning Transformer for Remote Sensing Visual Grounding

Visual Grounding With Joint Multimodal Representation and Interaction

Hierarchical Multi-Modality Graph Reasoning for Remote Sensing Visual Question Answering

SimVG: A Simple Framework for Visual Grounding with Decoupled Multi-modal Fusion

Visual Selection and Multistage Reasoning for RSVG

A multi-scale contextual attention network for remote sensing visual question answering

MFVNet: a deep adaptive fusion network with multiple field-of-views for remote sensing image semantic segmentation

A Fusion Encoder with Multi-Task Guidance for Cross-Modal Text–Image Retrieval in Remote Sensing

Semantic-Aligned Cross-Modal Visual Grounding Network with Transformers

Unified Transformer with Cross-Modal Mixture Experts for Remote-Sensing Visual Question Answering