Exploring Text-Guided Single Image Editing for Remote Sensing Images
Fangzhou Han,Lingyu Si,Hongwei Dong,Lamei Zhang,Hao Chen,Bo Du
2024-09-26
Abstract:Artificial intelligence generative content (AIGC) has significantly impacted image generation in the field of remote sensing. However, the equally important area of remote sensing image (RSI) editing has not received sufficient attention. Deep learning based editing methods generally involve two sequential stages: generation and editing. During the generation stage, consistency in content and details between the original and edited images must be maintained, while in the editing stage, controllability and accuracy of the edits should be ensured. For natural images, these challenges can be tackled by training generative backbones on large-scale benchmark datasets and using text guidance based on vision-language models (VLMs). However, these previously effective approaches become less viable for RSIs due to two reasons: First, existing generative RSI benchmark datasets do not fully capture the diversity of remote sensing scenarios, particularly in terms of variations in sensors, object types, and resolutions. Consequently, the generalization capacity of the trained backbone model is often inadequate for universal editing tasks on RSIs. Second, the large spatial resolution of RSIs exacerbates the problem in VLMs where a single text semantic corresponds to multiple image semantics, leading to the introduction of incorrect semantics when using text to guide RSI editing. To solve above problems, this paper proposes a text-guided RSI editing method that is controllable but stable, and can be trained using only a single image. It adopts a multi-scale training approach to preserve consistency without the need for training on extensive benchmark datasets, while leveraging RSI pre-trained VLMs and prompt ensembling (PE) to ensure accuracy and controllability in the text-guided editing process.
Computer Vision and Pattern Recognition