TourLLM: Enhancing LLMs with Tourism Knowledge

Qikai Wei,Mingzhi Yang,Jinqiang Wang,Wenwei Mao,Jiabo Xu,Huansheng Ning
2024-06-18
Abstract:Recently, large language models (LLMs) have demonstrated their effectiveness in various natural language processing (NLP) tasks. However, the lack of tourism knowledge limits the performance of LLMs in tourist attraction presentations and travel planning. To address this challenge, we constructed a supervised fine-tuning dataset for the culture and tourism domain, named Cultour. This dataset consists of three parts: tourism knowledge base QA data, travelogues data, and tourism diversity QA data. Additionally, we propose TourLLM, a Qwen-based model supervised fine-tuned with Cultour, to improve the quality of the information provided about attractions and travel planning. To evaluate the performance of TourLLM, we employed both automatic and human evaluation, and we proposed a human evaluation criterion named CRA (Consistency, Readability, Availability). The experimental results demonstrate the effectiveness of the responses generated by the TourLLM. Our proposed Cultour is accessible at <a class="link-external link-https" href="https://github.com/mrweiqk/Cultour" rel="external noopener nofollow">this https URL</a>.
Computation and Language,Artificial Intelligence
What problem does this paper attempt to address?
The main goal of this paper is to address the limitations of large language models (LLMs) in the tourism domain, particularly the lack of domain-specific knowledge, which results in limited performance in attractions recommendation and travel planning. To tackle this issue, the authors constructed a supervised fine-tuning dataset named Cultour, which includes three parts: tourism knowledge Q&A data, travelogue data, and tourism diversity Q&A data. Based on this dataset, the authors proposed a model named TourLLM, which is a supervised fine-tuned version of Qwen 1.5, aimed at improving the quality of information on tourist attractions and travel planning advice. Specifically, the contributions of the authors' work are as follows: 1. Constructed a high-quality Chinese supervised fine-tuning dataset, Cultour, specifically targeting the cultural and tourism domain. This dataset includes tourism knowledge base Q&A data, travelogue data, and tourism diversity Q&A data. 2. Proposed the TourLLM model, which is based on Qwen 1.5 and has been supervised fine-tuned using the Cultour dataset to improve the quality of information provided in the tourism domain. 3. Introduced a new evaluation metric, CRA (Consistency, Readability, Usability), for manually evaluating the performance of LLMs in the tourism domain. 4. Evaluated TourLLM through both automatic and manual evaluation methods, with experimental results demonstrating the effectiveness of TourLLM. In summary, this paper aims to enhance the performance of LLMs in the tourism domain through specialized datasets and model fine-tuning, to provide more accurate and richer tourism information and services.