Abstract:Remote Sensing Visual Question Answering (RSVQA) has gained significant research interest. However, current RSVQA methods are limited by the imaging mechanisms of optical sensors, particularly under challenging conditions such as cloud-covered and low-light scenarios. Given the all-time and all-weather imaging capabilities of Synthetic Aperture Radar (SAR), it is crucial to investigate the integration of optical-SAR images to improve RSVQA performance. In this work, we propose a Text-guided Coarse-to-Fine Fusion Network (TGFNet), which leverages the semantic relationships between question text and multi-source images to guide the network toward complementary fusion at the feature level. Specifically, we develop a Text-guided Coarse-to-Fine Attention Refinement (CFAR) module to focus on key areas related to the question in complex remote sensing images. This module progressively directs attention from broad areas to finer details through key region routing, enhancing the model's ability to focus on relevant regions. Furthermore, we propose an Adaptive Multi-Expert Fusion (AMEF) module that dynamically integrates different experts, enabling the adaptive fusion of optical and SAR features. In addition, we create the first large-scale benchmark dataset for evaluating optical-SAR RSVQA methods, comprising 6,008 well-aligned optical-SAR image pairs and 1,036,694 well-labeled question-answer pairs across 16 diverse question types, including complex relational reasoning questions. Extensive experiments on the proposed dataset demonstrate that our TGFNet effectively integrates complementary information between optical and SAR images, significantly improving the model's performance in challenging scenarios. The dataset is available at: <a class="link-external link-https" href="https://github.com/mmic-lcl/" rel="external noopener nofollow">this https URL</a>. Index Terms: Remote Sensing Visual Question Answering, Multi-source Data Fusion, Multimodal, Remote Sensing, OPT-SAR.

See, Perceive, and Answer: A Unified Benchmark for High-Resolution Postdisaster Evaluation in Remote Sensing Images

RescueADI: Adaptive Disaster Interpretation in Remote Sensing Images with Autonomous Agents

VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understanding

RSVQA: Visual Question Answering for Remote Sensing Data

Unleashing the Potential of Large Language Model: Zero-shot VQA for Flood Disaster Scenario

Multilingual Augmentation for Robust Visual Question Answering in Remote Sensing Images

MS4D-Net: Multitask-Based Semi-Supervised Semantic Segmentation Framework with Perturbed Dual Mean Teachers for Building Damage Assessment from High-Resolution Remote Sensing Imagery

Enhancing Remote Sensing Visual Question Answering: A Mask-Based Dual-Stream Feature Mutual Attention Network

Text-Guided Coarse-to-Fine Fusion Network for Robust Remote Sensing Visual Question Answering

Segmentation-guided Attention for Visual Question Answering from Remote Sensing Images

Large Vision-Language Models for Remote Sensing Visual Question Answering

RSGPT: A Remote Sensing Vision Language Model and Benchmark

A Method of Efficient Synthesizing Post-disaster Remote Sensing Image with Diffusion Model and LLM

Panoptic Perception: A Novel Task and Fine-grained Dataset for Universal Remote Sensing Image Interpretation

A Deep Multitask Convolutional Neural Network for Remote Sensing Image Super-Resolution and Colorization

Multi-Scale and Context-Aware Framework for Flood Segmentation in Post-Disaster High Resolution Aerial Images

EMR-HRNet: A Multi-Scale Feature Fusion Network for Landslide Segmentation from Remote Sensing Images

SAM-VQA: Supervised Attention-Based Visual Question Answering Model for Post-Disaster Damage Assessment on Remote Sensing Imagery

COREval: A Comprehensive and Objective Benchmark for Evaluating the Remote Sensing Capabilities of Large Vision-Language Models

FloodNet: A High Resolution Aerial Imagery Dataset for Post Flood Scene Understanding

A Spatial Hierarchical Reasoning Network for Remote Sensing Visual Question Answering