Functional requirements to mitigate the Risk of Harm to Patients from Artificial Intelligence in Healthcare

Juan M. García-Gómez,Vicent Blanes-Selva,José Carlos de Bartolomé Cenzano,Jaime Cebolla-Cornejo,Ascensión Doñate-Martínez
2023-09-19
Abstract:The Directorate General for Parliamentary Research Services of the European Parliament has prepared a report to the Members of the European Parliament where they enumerate seven main risks of Artificial Intelligence (AI) in medicine and healthcare: patient harm due to AI errors, misuse of medical AI tools, bias in AI and the perpetuation of existing inequities, lack of transparency, privacy and security issues, gaps in accountability, and obstacles in implementation. In this study, we propose fourteen functional requirements that AI systems may implement to reduce the risks associated with their medical purpose: AI passport, User management, Regulation check, Academic use only disclaimer, data quality assessment, Clinicians double check, Continuous performance evaluation, Audit trail, Continuous usability test, Review of retrospective/simulated cases, Bias check, eXplainable AI, Encryption and use of field-tested libraries, and Semantic interoperability. Our intention here is to provide specific high-level specifications of technical solutions to ensure continuous good performance and use of AI systems to benefit patients in compliance with the future EU regulatory framework.
Artificial Intelligence
What problem does this paper attempt to address?
This paper discusses the risks that artificial intelligence (AI) may bring to healthcare and proposes a set of functional requirements to mitigate these risks. The paper lists seven risks identified by the European Parliament's Research Services, including patient harm caused by AI errors, misuse of medical AI tools, persistent AI bias and inequality, lack of transparency, privacy and security issues, lack of accountability, and implementation barriers. To address these risks, the paper proposes 14 functional requirements, such as an AI passport, user management, regulatory checks, data quality assessment, and clinical double-check, aiming to ensure the continued good performance and compliant use of AI systems in the medical field, while aligning with future EU regulatory frameworks. In addition, the paper demonstrates how these risk mitigation functional requirements can be applied through a case of AI-assisted palliative care decision-making.