ScribbleEditor: Guided Photo-realistic and Identity-preserving Image Editing with Interactive Scribble

Haotian Hu,Bin Jiang,Chao Yang,Xinjiao Zhou,Xiaofei Huo
DOI: https://doi.org/10.1109/icme57554.2024.10688177
2024-01-01
Abstract:Free-form scribbles are a convenient way for users to describe their intentions for image editing. However, prevalent issues arise in existing methods. First, the rich color of scribbles may contain information that deviates significantly from the original image distribution, causing common blending methods to yield unrealistic results due to inadequate interaction between the scribble and the image. Secondly, inputting extensive scribble areas may obscure crucial image components, resulting in identity loss post-editing. To address these issues, we propose a two-stage scribble editing method that achieves photo-realistic and identity-preserving editing. Our proposed method employs a two-channel multimodal interaction module, facilitating deep interaction between scribbles and images in channel and position domains, achieving semantic alignment and enhancement. In the second stage, we interact the editing offset with scribble area image content via the texture supplement module to achieve texture detail supplementation. Our method adeptly handles complex scribbles, even across extensive areas, demonstrating superior photo-realism and identity preservation in experimental results.
What problem does this paper attempt to address?