Abstract:Remote Sensing Visual Question Answering (RSVQA) has gained significant research interest. However, current RSVQA methods are limited by the imaging mechanisms of optical sensors, particularly under challenging conditions such as cloud-covered and low-light scenarios. Given the all-time and all-weather imaging capabilities of Synthetic Aperture Radar (SAR), it is crucial to investigate the integration of optical-SAR images to improve RSVQA performance. In this work, we propose a Text-guided Coarse-to-Fine Fusion Network (TGFNet), which leverages the semantic relationships between question text and multi-source images to guide the network toward complementary fusion at the feature level. Specifically, we develop a Text-guided Coarse-to-Fine Attention Refinement (CFAR) module to focus on key areas related to the question in complex remote sensing images. This module progressively directs attention from broad areas to finer details through key region routing, enhancing the model's ability to focus on relevant regions. Furthermore, we propose an Adaptive Multi-Expert Fusion (AMEF) module that dynamically integrates different experts, enabling the adaptive fusion of optical and SAR features. In addition, we create the first large-scale benchmark dataset for evaluating optical-SAR RSVQA methods, comprising 6,008 well-aligned optical-SAR image pairs and 1,036,694 well-labeled question-answer pairs across 16 diverse question types, including complex relational reasoning questions. Extensive experiments on the proposed dataset demonstrate that our TGFNet effectively integrates complementary information between optical and SAR images, significantly improving the model's performance in challenging scenarios. The dataset is available at: <a class="link-external link-https" href="https://github.com/mmic-lcl/" rel="external noopener nofollow">this https URL</a>. Index Terms: Remote Sensing Visual Question Answering, Multi-source Data Fusion, Multimodal, Remote Sensing, OPT-SAR.

A Spatial Frequency Fusion Strategy Based on Linguistic Query Refinement for RSVG

Visual Selection and Multistage Reasoning for RSVG

Injecting Linguistic into Visual Backbone: Query-Aware Multimodal Fusion Network for Remote Sensing Visual Grounding

Language-Guided Progressive Attention for Visual Grounding in Remote Sensing Images

Language Query-Based Transformer With Multiscale Cross-Modal Alignment for Visual Grounding on Remote Sensing Images

Language Query Based Transformer with Multi-Scale Cross-Modal Alignment for Visual Grounding on Remote Sensing Images

Exploring Fine-Grained Image-Text Alignment for Referring Remote Sensing Image Segmentation

A Frequency Domain Feature-Guided Network for Semantic Segmentation of Remote Sensing Images

Text-Guided Coarse-to-Fine Fusion Network for Robust Remote Sensing Visual Question Answering

ResVG: Enhancing Relation and Semantic Understanding in Multiple Instances for Visual Grounding

Spatial Semantic Recurrent Mining for Referring Image Segmentation

Multi-Stage Synergistic Aggregation Network for Remote Sensing Visual Grounding

Fine-Grained Information Supplementation and Value-Guided Learning for Remote Sensing Image-Text Retrieval

Mutual Attention Inception Network for Remote Sensing Visual Question Answering

A Cross-Modal Semantic Mapping Enhancement Model for Remote Sensing Visual Question Answering

An End-to-End Framework Based on Vision-Language Fusion for Remote Sensing Cross-Modal text-Image Retrieval

Hierarchical Multi-Modality Graph Reasoning for Remote Sensing Visual Question Answering

Attribute-Prompting Multi-Modal Object Reasoning Transformer for Remote Sensing Visual Grounding

A multi-scale contextual attention network for remote sensing visual question answering

VSDF: A variation-based spatiotemporal data fusion method

OPTICAL AND SAR IMAGE FUSION BASED ON VISUAL SALIENCY FEATURES