The Elements Extraction on Traditional Chinese Paintings Based on Object Detection

Qingyu Meng,Kaiyue Li,Mingquan Zhou,Huanhuan Zhang
DOI: https://doi.org/10.1145/3375959.3375980
2019-01-01
Abstract:The traditional Chinese painting has great cultural and historic value for us. Because of that, the research on the paintings has never stopped. During the current research stage, we would like to design a system which can rapidly construct a 3D scene according to a traditional Chinese painting. In the previous article [1], we designed a convolutional neural network referenced to VGG-16 [2] to classify traditional Chinese paintings into figure painting, landscape painting, and flower-and-bird painting according to the content of the paintings. However, it is not enough to be able to classify a painting. If we can quickly extract the main elements of a painting, it will greatly facilitate the construction of a 3D scene. To this end, we have studied the object detection algorithm based on deep learning which achieves amazing accuracy on a natural image dataset and tried to migrate it to the traditional Chinese painting dataset. Considering that our construction system may be applied on the mobile side like a smartphone, we used two algorithms which are faster and less computationally intensive: YOLOv3 and RetinaNet. According to the characteristics of traditional Chinese painting dataset, the network structure and some hyper-parameters were modified. Finally, we experimentally proved the effectiveness of such algorithms in the traditional Chinese painting application scenarios.
What problem does this paper attempt to address?