Construction of Multimodal Dialog System Via Knowledge Graph in Travel Domain

Jing Wan,Minghui Yuan,Zhenhao Dong,Lei Hou,Jiawang Xie,Hongyin Zhu,Qinghua Wen
DOI: https://doi.org/10.1007/978-981-97-2421-5_28
2024-01-01
Abstract:When traveling to a foreign city, we often find ourselves in dire need of an intelligent agent that can provide instant and informative responses to our various queries. Such an agent should have the ability to understand our queries and possess the knowledge to generate helpful responses. Furthermore, if the agent can comprehend image information, it can provide solutions from multiple perspectives. Knowledge graph-based multimodal dialog systems offer a promising approach to fulfill these requirements. In this paper, we present a solution for efficiently constructing a multimodal dialog system in the travel domain without large-scale datasets. The system’s main objective is to assist users in completing various travel-related tasks, specifically attraction recommendation and route planning, which are frequently requested by users while traveling. We introduce the Multimodal Chinese Tourism Knowledge Graph (MCTKG) and integrate image processing and recommendation technology into a dialog system. Specifically, our approach utilizes modular design to construct the dialog system, and leverages the rich information available in the knowledge graph to enhance the performance of each module. To the best of our knowledge, this is the first multimodal travel dialog system that provides users with personalized travel route recommendations. Multiple experiments have proven that our dialog system can effectively enhance the user’s travel experience.
What problem does this paper attempt to address?