SceneSketcher: Fine-Grained Image Retrieval with Scene Sketches
Fang Liu,Changqing Zou,Xiaoming Deng,Ran Zuo,Yu-Kun Lai,Cuixia Ma,Yong-Jin Liu,Hongan Wang
DOI: https://doi.org/10.1007/978-3-030-58529-7_42
2020-01-01
Abstract:Sketch-based image retrieval (SBIR) has been a popular research topic in recent years. Existing works concentrate on mapping the visual information of sketches and images to a semantic space at the object level. In this paper, for the first time, we study the fine-grained scene-level SBIR problem which aims at retrieving scene images satisfying the user’s specific requirements via a freehand scene sketch. We propose a graph embedding based method to learn the similarity measurement between images and scene sketches, which models the multi-modal information, including the size and appearance of objects as well as their layout information, in an effective manner. To evaluate our approach, we collect a dataset based on SketchyCOCO and extend the dataset using Coco-stuff. Comprehensive experiments demonstrate the significant potential of the proposed approach on the application of fine-grained scene-level image retrieval.