Construction and Application of Multi-modal Knowledge Graph for Collection of Cultural Relics

Bowei Liu,Wanqing Zhao,Jun Wang,Jing Yan,Jinye Peng

DOI: https://doi.org/10.1109/icipmc58929.2023.00030

2023-01-01

Abstract:In recent years, computer technology has advanced significantly and more and more museums have begun to prioritize digital protection of cultural relics, achieving fruitful results. However, the digital resources of these relics are often stored in document or relational database forms, and displayed on the museum website using basic division methods. Moreover, each cultural relic collection holds its own unique knowledge points and there must be rich associations between them due to certain commonalities, which cannot be fully applied by existing preservation techniques. Multi-modal Knowledge Graphs (MMKG) have undergone years of development and have achieved significant results and widespread applications. Although China has abundant multi-modal data on cultural relics, there is limited exploration of MMKG in the field of cultural relics. Therefore, this paper proposes to construct a MMKG for museum relics, RelicMMKG, to mine the information contained in the multimodal digital resources of museum relics. This includes using the BERT-BiLSTM-CRF model to identify multiple types of entities in the relic text introduction and using image descriptors to extract and calculate the similarity of visual features in the relic image data. Furthermore, based on the RelicMMKG and structured text and image data, a multifunctional application system is implemented to better explore the correlation between cultural relics, enabling efficient excavation and application of cultural relics data in the museum.