Sketch based image retrieval based on abstract-level transform and convolutional neural networks

Yujie LIU,Yunping PANG,Zongmin LI,Hua LI
DOI: https://doi.org/10.3785/j.issn.1008-9497.2016.06.005
2016-01-01
Abstract:The traditional methods on sketch based image retrieval (SBIR) is mainly based on the hand-crafted de-scriptors such as HOG and SIFT .Considering the limitations of the traditional hand-crafted descriptors ,we propose a novel approach based on the abstract-level transform and the convolutional neural network (CNN) .Our work is re-alized by the following steps :1) Extracting the boundary probability images from the database images ;2) Conver-ting the boundary probability images into abstract-level images ;3) Inputting the abstract-level images into the net-works and extracting the hidden layers' output vectors ;4) Combining different hidden layers' output vectors as the final descriptor for retrieval .We evaluate our proposed retrieval strategy on Flickr15K datasets .The main contribu-tions of our work are the preprocessing based on the boundary probability detector and the abstract-level transform ation ,furthermore ,proposing an improved combination of deep features .Results show that the proposal achieves significant improvements .
What problem does this paper attempt to address?