Explainable AI applications in the Medical Domain: a systematic review

Nicoletta Prentzas,Antonis Kakas,Constantinos S. Pattichis
DOI: https://doi.org/10.48550/arXiv.2308.05411
2023-08-10
Abstract:Artificial Intelligence in Medicine has made significant progress with emerging applications in medical imaging, patient care, and other areas. While these applications have proven successful in retrospective studies, very few of them were applied in practice.The field of Medical AI faces various challenges, in terms of building user trust, complying with regulations, using data ethically.Explainable AI (XAI) aims to enable humans understand AI and trust its results. This paper presents a literature review on the recent developments of XAI solutions for medical decision support, based on a representative sample of 198 articles published in recent years. The systematic synthesis of the relevant articles resulted in several findings. (1) model-agnostic XAI techniques were mostly employed in these solutions, (2) deep learning models are utilized more than other types of machine learning models, (3) explainability was applied to promote trust, but very few works reported the physicians participation in the loop, (4) visual and interactive user interface is more useful in understanding the explanation and the recommendation of the system. More research is needed in collaboration between medical and AI experts, that could guide the development of suitable frameworks for the design, implementation, and evaluation of XAI solutions in medicine.
Artificial Intelligence,Machine Learning
What problem does this paper attempt to address?
The main problem this paper attempts to address is the interpretability of artificial intelligence (AI) systems in the medical field. Specifically: 1. **Trust and Transparency**: Many current medical AI systems perform well in retrospective studies but face challenges in real-world applications, particularly in establishing user trust, complying with regulations, and ethically using data. 2. **Application of Explainable AI (XAI)**: By introducing explainable AI technologies to enhance doctors' trust in AI systems, enabling them to understand the reasons behind AI decisions. 3. **Collaboration between Technologists and Medical Experts**: Emphasizes the importance of collaboration between medical experts and AI experts in developing XAI solutions suitable for medical practice. 4. **Evaluation Methods**: The paper also explores how to evaluate the effectiveness of these XAI technologies and proposes directions for further research, including how to design, implement, and evaluate XAI systems applicable to medical scenarios. In summary, through a systematic review of relevant literature in recent years, this paper aims to summarize the current application status of existing XAI technologies in medical decision support systems and highlight key directions for future research.