Abstract:The popularity of information technology has given rise to a growing interest in smart education and has provided the possibility of combining online and offline education. Knowledge graphs, an effective technology for knowledge representation and management, have been successfully utilized to manage massive educational resources. However, the existing research on constructing educational knowledge graphs ignores multiple modalities and their relationships, such as teacher speeches and their relationship with knowledge. To tackle this problem, we propose an automatic approach to construct multi-modal educational knowledge graphs that integrate speech as a modal resource to facilitate the reuse of educational resources. Specifically, we first propose a fine-tuned Bidirectional Encoder Representation from Transformers (BERT) model based on education lexicon, called EduBERT, which can adaptively capture effective information in the education field. We also add a Bidirectional Long Short-Term Memory-Conditional Random Field (BiLSTM-CRF) to effectively identify educational entities. Then, the locational information of the entity is incorporated into BERT to extract the educational relationship. In addition, to cover the shortage of traditional text-based knowledge graphs, we focus on collecting teacher speech to construct a multi-modal knowledge graph. We propose a speech-fusion method that links these data into the graph as a class of entities. The numeric results show that our proposed approach can manage and present various modes of educational resources and that it can provide better education services.

A Multi-Modal Knowledge Graph for Classical Chinese Poetry.

GeLaiGeLai: A visual platform for analysis of Classical Chinese Poetry based on Knowledge Graph

WuMKG: a Chinese painting and calligraphy multimodal knowledge graph

A Multi-Modal Chinese Poetry Generation Model

MMPKUBase: A Comprehensive and High-quality Chinese Multi-modal Knowledge Graph

Multimodal knowledge graph construction of Chinese traditional operas and sentiment and genre recognition

TopWORDS-Poetry: Simultaneous Text Segmentation and Word Discovery for Classical Chinese Poetry Via Bayesian Inference.

MMpedia: A Large-Scale Multi-modal Knowledge Graph

Deep Poetry: A Chinese Classical Poetry Generation System

GPT-based Generation for Classical Chinese Poetry

Images2Poem: Generating Chinese Poetry from Image Streams.

Generating Major Types of Chinese Classical Poetry in a Uniformed Framework

CharPoet: A Chinese Classical Poetry Generation System Based on Token-free LLM

iPoet: interactive painting poetry creation with visual multimodal analysis

A method of constructing a fine-grained sentiment lexicon for the humanities computing of classical chinese poetry

MEduKG: A Deep-Learning-Based Approach for Multi-Modal Educational Knowledge Graph Construction

PKGM: A Pre-trained Knowledge Graph Model for E-commerce Application

Poetic Expression Through Scenery: Sentimental Chinese Classical Poetry Generation from Images.

Multi-Modal Knowledge Graph Construction and Application: A Survey

Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey

CCPM: A Chinese Classical Poetry Matching Dataset