Chinese Diabetes Question Classification Using Large Language Models and Transfer Learning

Chengze Ge,Hongshun Ling,Fuliang Quan,Jianping Zeng

DOI: https://doi.org/10.1007/978-981-97-1717-0_19

2024-01-01

Abstract:Type 2 diabetes has evolved into a significant global public health challenge. Diabetes question-answering services are playing an increasingly important role in providing daily health services for patients and high-risk populations. As one of the evaluation track for CHIP 2023, participants are required to classify diabetes-related questions. We have introduced an approach that utilizes generative open-source large language models to accomplish this task. Initially, we designed a prompt construction method that transforms question-label pairs into a conversational text. Subsequently, we fine-tuned the large language model using LoRA method. Furthermore, to enhance the capability in the medical domain, we employed another open-source dataset for initial fine-tuning of the model, followed by transfer learning to fine-tune the Chinese diabetes questions dataset. Experimental results demonstrate the superiority of our approach, ultimately achieving a score of 92.10 on the test data.