A Sketch Recognition Method Based on Transfer Deep Learning with the Fusion of Multi-Granular Sketches

Peng Zhao,Yang Liu,Yijuan Lu,Benpeng Xu
DOI: https://doi.org/10.1007/s11042-019-08216-6
IF: 2.577
2019-01-01
Multimedia Tools and Applications
Abstract:Most of existing sketch recognition methods focus on the contour/shape of whole sketches. They ignore different granularities of sketches during sketching. Stroke sequences of sketches often demonstrate the change of various granularities. In the progress of sketching, a coarser-grained contour gradually changes to a finer-grained object. Different granularities of sketch imply different levels of semantic information and play different roles in sketch recognition. In this paper, a transfer-deep-learning-based sketch recognition method-"sketch-transfer-net" is proposed. Sketch-transfer-net designs a novel fine-tuning strategy to use different granular sketches to fine-tune different layers of neural network. The extensive comparative experiments show that the proposed sketch-transfer-net can capture descriptive information of various granular sketches and therefore improve the performance of sketch recognition. In addition, the novel fine-turning strategy could weaken the negative effect in transfer learning and enable CNNs to be well trained on small sketch datasets.
What problem does this paper attempt to address?