Quality of Information Provided by Artificial Intelligence Chatbots Surrounding the Reconstructive Surgery for Head and Neck Cancer: A Comparative Analysis Between ChatGPT4 and Claude2
Paolo Boscolo‐Rizzo,Alberto Vito Marcuzzo,Chiara Lazzarin,Fabiola Giudici,Jerry Polesel,Marco Stellin,Andrea Pettorelli,Giacomo Spinato,Giancarlo Ottaviano,Marco Ferrari,Daniele Borsetto,Simone Zucchini,Franco Trabalzini,Egidio Sia,Nicoletta Gardenal,Roberto Baruca,Alfonso Fortunati,Luigi Angelo Vaira,Giancarlo Tirelli
DOI: https://doi.org/10.1111/coa.14261
IF: 2.729
2024-12-06
Clinical Otolaryngology
Abstract:Introduction Artificial Intelligences (AIs) are changing the way information is accessed and consumed globally. This study aims to evaluate the information quality provided by AIs ChatGPT4 and Claude2 concerning reconstructive surgery for head and neck cancer. Methods Thirty questions on reconstructive surgery for head and neck cancer were directed to both AIs and 16 head and neck surgeons assessed the responses using the QAMAI questionnaire. A 5‐point Likert scale was used to assess accuracy, clarity, relevance, completeness, sources, and usefulness. Questions were categorised into those suitable for patients (group 1) and those for surgeons (group 2). AI responses were compared using t‐Student and McNemar tests. Surgeon score agreement was measured with intraclass correlation coefficient, and readability was assessed with Flesch–Kincaid Grade Level (FKGL). Results ChatGPT4 and Claude2 had similar overall mean scores of accuracy, clarity, relevance, completeness and usefulness, while Claude2 outperformed ChatGPT4 in sources (110.0 vs. 92.1, p
otorhinolaryngology