Abstract:Background: Large language models (LLMs) have achieved great progress in natural language processing tasks and demonstrated the potential for use in clinical applications. Despite their capabilities, LLMs in the medical domain are prone to generating hallucinations (not fully reliable responses). Hallucinations in LLMs' responses create substantial risks, potentially threatening patients' physical safety. Thus, to perceive and prevent this safety risk, it is essential to evaluate LLMs in the medical domain and build a systematic evaluation. Objective: We developed a comprehensive evaluation system, MedGPTEval, composed of criteria, medical data sets in Chinese, and publicly available benchmarks. Methods: First, a set of evaluation criteria was designed based on a comprehensive literature review. Second, existing candidate criteria were optimized by using a Delphi method with 5 experts in medicine and engineering. Third, 3 clinical experts designed medical data sets to interact with LLMs. Finally, benchmarking experiments were conducted on the data sets. The responses generated by chatbots based on LLMs were recorded for blind evaluations by 5 licensed medical experts. The evaluation criteria that were obtained covered medical professional capabilities, social comprehensive capabilities, contextual capabilities, and computational robustness, with 16 detailed indicators. The medical data sets include 27 medical dialogues and 7 case reports in Chinese. Three chatbots were evaluated: ChatGPT by OpenAI; ERNIE Bot by Baidu, Inc; and Doctor PuJiang (Dr PJ) by Shanghai Artificial Intelligence Laboratory. Results: Dr PJ outperformed ChatGPT and ERNIE Bot in the multiple-turn medical dialogues and case report scenarios. Dr PJ also outperformed ChatGPT in the semantic consistency rate and complete error rate category, indicating better robustness. However, Dr PJ had slightly lower scores in medical professional capabilities compared with ChatGPT in the multiple-turn dialogue scenario. Conclusions: MedGPTEval provides comprehensive criteria to evaluate chatbots by LLMs in the medical domain, open-source data sets, and benchmarks assessing 3 LLMs. Experimental results demonstrate that Dr PJ outperforms ChatGPT and ERNIE Bot in social and professional contexts. Therefore, such an assessment system can be easily adopted by researchers in this community to augment an open-source data set.

LLM-Mini-CEX: Automatic Evaluation of Large Language Model for Diagnostic Conversation

TCMChat: A Generative Large Language Model for Traditional Chinese Medicine

Towards Automatic Evaluation for LLMs' Clinical Capabilities: Metric, Data, and Algorithm

An Automatic Evaluation Framework for Multi-turn Medical Consultations Capabilities of Large Language Models

AI Hospital: Interactive Evaluation and Collaboration of LLMs As Intern Doctors for Clinical Diagnosis

Improving Clinical Expertise in Large Language Models Using Electronic Medical Records

Automatic Interactive Evaluation for Large Language Models with State Aware Patient Simulator

MEDIC: Towards a Comprehensive Framework for Evaluating LLMs in Clinical Applications

PALLM: Evaluating and Enhancing PALLiative Care Conversations with Large Language Models

Data Set and Benchmark (MedGPTEval) to Evaluate Responses From Large Language Models in Medicine: Evaluation Development and Validation

CliBench: A Multifaceted and Multigranular Evaluation of Large Language Models for Clinical Decision Making

Evaluating large language models in medical applications: a survey

GameEval: Evaluating LLMs on Conversational Games

CliMedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models in Clinical Scenarios

AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator

Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries

Leveraging Large Language Model as Simulated Patients for Clinical Education

MedKP: Medical Dialogue with Knowledge Enhancement and Clinical Pathway Encoding

A Novel Nuanced Conversation Evaluation Framework for Large Language Models in Mental Health

Testing the Limits of Language Models: A Conversational Framework for Medical AI Assessment

The Potential of LLMs in Medical Education: Generating Questions and Answers for Qualification Exams