Evaluating Chatgpt As an Adjunct for Analyzing Challenging Case
Xiaoni Liu,Ying Song,Hai Lin,Yiying Xu,Chao Chen,Changjian Yan,Xiaoliang Yuan,Fang Bao,Hongmei Jing,Chaoling Wu,Weilong Zhang
DOI: https://doi.org/10.1182/blood-2023-181518
IF: 20.3
2023-01-01
Blood
Abstract:Introduction Despite the rapid development of medicine, clinicians still faced with many challenging cases in practice. For challenging cases, most inexperienced doctors are unable to make a clear diagnosis in the first place, resulting in misdiagnosis and delayed disease. Recently, OpenAI released a new Chatbot model, called ChatGPT (Generation Pre training Converter), which is an artificial intelligence (AI) system that uses reinforcement learning model based on human feedback for training. There have been many research reports on the application of ChatGPT in the medical field, and it has been proven to have a wide range of clinical applications. In our study, we aimed to assess the accuracy and reliability of ChatGPT in diagnosing and treating challenging case. Methods We provided 2 challenging cases. Case 1 provided a medical history and auxiliary test results for ChatGPT, and then ChatGPT answered the possible diagnosis, diagnostic basis, further auxiliary examination and treatment. Case 2 provides negative symptoms and auxiliary test results for ChatGPT, which answers all possible diagnoses. Conduct a comprehensive evaluation and scoring of the results of ChatGPT responses by 5 senior physicians. A score of 10 is the perfect score. Results The analysis results of 2 challenging cases by ChatGPT were comprehensively evaluated by senior experienced professional doctors and scored 9 points. We found that ChatGPT's analysis of challenging case is comprehensive. Especially for the diagnosis and treatment of diseases, ChatGPT analyzes all possible diagnoses and provides accurate treatment plans through the provided medical history, auxiliary examination results, and treatment processes. ChatGPT can directly interpret the auxiliary examination results, such as high level of procalcitonin (PCT), abnormal kidney function, and with negative results can rule out certain diseases, such as ANA, ENA, and ANCA negative, ruling out the possibility of autoimmune diseases. ChatGPT can summarize all the diagnoses including rare diseases, based on the results of an auxiliary test, and explain the cause of the disease and the pathogenesis in detail. Conclusions Our study shows that ChatGPT can provide important reference value as an adjunct to clinicians in challenging case analysis. In particular, it can assist inexperienced doctors in remote areas and underdeveloped medical conditions to make clear diagnosis and treatment as soon as possible, avoid unnecessary referral to higher-level hospitals, reduce medical burden, and reduce waste of medical resources. In the medical field, ChatGPT needs more research to evaluate in clinical applications and become a better adjunct tool in clinical work.