TICAS assessment of cells from atypical hyperplasia of the endometrium.

G. Wied,P. Bartels,G. F. Bahr,J. W. Reagan

1969-10-01

Acta Cytologica

Abstract:

What problem does this paper attempt to address?

An Explainable AI Framework for Artificial Intelligence of Medical Things

Al Amin,Kamrul Hasan,Saleh Zein-Sabatto,Deo Chimba,Imtiaz Ahmed,Tariqul Islam

2024-03-07

Abstract:The healthcare industry has been revolutionized by the convergence of Artificial Intelligence of Medical Things (AIoMT), allowing advanced data-driven solutions to improve healthcare systems. With the increasing complexity of Artificial Intelligence (AI) models, the need for Explainable Artificial Intelligence (XAI) techniques become paramount, particularly in the medical domain, where transparent and interpretable decision-making becomes crucial. Therefore, in this work, we leverage a custom XAI framework, incorporating techniques such as Local Interpretable Model-Agnostic Explanations (LIME), SHapley Additive exPlanations (SHAP), and Gradient-weighted Class Activation Mapping (Grad-Cam), explicitly designed for the domain of AIoMT. The proposed framework enhances the effectiveness of strategic healthcare methods and aims to instill trust and promote understanding in AI-driven medical applications. Moreover, we utilize a majority voting technique that aggregates predictions from multiple convolutional neural networks (CNNs) and leverages their collective intelligence to make robust and accurate decisions in the healthcare system. Building upon this decision-making process, we apply the XAI framework to brain tumor detection as a use case demonstrating accurate and transparent diagnosis. Evaluation results underscore the exceptional performance of the XAI framework, achieving high precision, recall, and F1 scores with a training accuracy of 99% and a validation accuracy of 98%. Combining advanced XAI techniques with ensemble-based deep-learning (DL) methodologies allows for precise and reliable brain tumor diagnoses as an application of AIoMT.

Computer Vision and Pattern Recognition
Explainable Machine Learning-Based Security and Privacy Protection Framework for Internet of Medical Things Systems

Ayoub Si-ahmed,Mohammed Ali Al-Garadi,Narhimene Boustia

2024-03-14

Abstract:The Internet of Medical Things (IoMT) transcends traditional medical boundaries, enabling a transition from reactive treatment to proactive prevention. This innovative method revolutionizes healthcare by facilitating early disease detection and tailored care, particularly in chronic disease management, where IoMT automates treatments based on real-time health data collection. Nonetheless, its benefits are countered by significant security challenges that endanger the lives of its users due to the sensitivity and value of the processed data, thereby attracting malicious interests. Moreover, the utilization of wireless communication for data transmission exposes medical data to interception and tampering by cybercriminals. Additionally, anomalies may arise due to human errors, network interference, or hardware malfunctions. In this context, anomaly detection based on Machine Learning (ML) is an interesting solution, but it comes up against obstacles in terms of explicability and protection of privacy. To address these challenges, a new framework for Intrusion Detection Systems (IDS) is introduced, leveraging Artificial Neural Networks (ANN) for intrusion detection while utilizing Federated Learning (FL) for privacy preservation. Additionally, eXplainable Artificial Intelligence (XAI) methods are incorporated to enhance model explanation and interpretation. The efficacy of the proposed framework is evaluated and compared with centralized approaches using multiple datasets containing network and medical data, simulating various attack types impacting the confidentiality, integrity, and availability of medical and physiological data. The results obtained offer compelling evidence that the FL method performs comparably to the centralized method, demonstrating high performance. Additionally, it affords the dual advantage of safeguarding privacy and providing model explanation.

Cryptography and Security,Artificial Intelligence
Unveiling the black box: A systematic review of Explainable Artificial Intelligence in medical image analysis

Dost Muhammad,Malika Bendechache

DOI: https://doi.org/10.1016/j.csbj.2024.08.005

2024-08-12

Abstract:This systematic literature review examines state-of-the-art Explainable Artificial Intelligence (XAI) methods applied to medical image analysis, discussing current challenges and future research directions, and exploring evaluation metrics used to assess XAI approaches. With the growing efficiency of Machine Learning (ML) and Deep Learning (DL) in medical applications, there's a critical need for adoption in healthcare. However, their "black-box" nature, where decisions are made without clear explanations, hinders acceptance in clinical settings where decisions have significant medicolegal consequences. Our review highlights the advanced XAI methods, identifying how they address the need for transparency and trust in ML/DL decisions. We also outline the challenges faced by these methods and propose future research directions to improve XAI in healthcare. This paper aims to bridge the gap between cutting-edge computational techniques and their practical application in healthcare, nurturing a more transparent, trustworthy, and effective use of AI in medical settings. The insights guide both research and industry, promoting innovation and standardisation in XAI implementation in healthcare.
Explainable Artificial Intelligence for Medical Applications: A Review

Qiyang Sun,Alican Akman,Björn W. Schuller

2024-11-15

Abstract:The continuous development of artificial intelligence (AI) theory has propelled this field to unprecedented heights, owing to the relentless efforts of scholars and researchers. In the medical realm, AI takes a pivotal role, leveraging robust machine learning (ML) algorithms. AI technology in medical imaging aids physicians in X-ray, computed tomography (CT) scans, and magnetic resonance imaging (MRI) diagnoses, conducts pattern recognition and disease prediction based on acoustic data, delivers prognoses on disease types and developmental trends for patients, and employs intelligent health management wearable devices with human-computer interaction technology to name but a few. While these well-established applications have significantly assisted in medical field diagnoses, clinical decision-making, and management, collaboration between the medical and AI sectors faces an urgent challenge: How to substantiate the reliability of decision-making? The underlying issue stems from the conflict between the demand for accountability and result transparency in medical scenarios and the black-box model traits of AI. This article reviews recent research grounded in explainable artificial intelligence (XAI), with an emphasis on medical practices within the visual, audio, and multimodal perspectives. We endeavour to categorise and synthesise these practices, aiming to provide support and guidance for future researchers and healthcare professionals.

Machine Learning,Computer Vision and Pattern Recognition
Transparency and Trust in Human-AI-Interaction: The Role of Model-Agnostic Explanations in Computer Vision-Based Decision Support

Christian Meske,Enrico Bunde

DOI: https://doi.org/10.1007/978-3-030-50334-5_4

2020-07-13

Abstract:Computer Vision, and hence Artificial Intelligence-based extraction of information from images, has increasingly received attention over the last years, for instance in medical diagnostics. While the algorithms' complexity is a reason for their increased performance, it also leads to the "black box" problem, consequently decreasing trust towards AI. In this regard, "Explainable Artificial Intelligence" (XAI) allows to open that black box and to improve the degree of AI transparency. In this paper, we first discuss the theoretical impact of explainability on trust towards AI, followed by showcasing how the usage of XAI in a health-related setting can look like. More specifically, we show how XAI can be applied to understand why Computer Vision, based on deep learning, did or did not detect a disease (malaria) on image data (thin blood smear slide images). Furthermore, we investigate, how XAI can be used to compare the detection strategy of two different deep learning models often used for Computer Vision: Convolutional Neural Network and Multi-Layer Perceptron. Our empirical results show that i) the AI sometimes used questionable or irrelevant data features of an image to detect malaria (even if correctly predicted), and ii) that there may be significant discrepancies in how different deep learning models explain the same prediction. Our theoretical discussion highlights that XAI can support trust in Computer Vision systems, and AI systems in general, especially through an increased understandability and predictability.

Human-Computer Interaction
A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI

Erico Tjoa,Cuntai Guan

DOI: https://doi.org/10.1109/TNNLS.2020.3027314

2020-08-11

Abstract:Recently, artificial intelligence and machine learning in general have demonstrated remarkable performances in many tasks, from image processing to natural language processing, especially with the advent of deep learning. Along with research progress, they have encroached upon many different fields and disciplines. Some of them require high level of accountability and thus transparency, for example the medical sector. Explanations for machine decisions and predictions are thus needed to justify their reliability. This requires greater interpretability, which often means we need to understand the mechanism underlying the algorithms. Unfortunately, the blackbox nature of the deep learning is still unresolved, and many machine decisions are still poorly understood. We provide a review on interpretabilities suggested by different research works and categorize them. The different categories show different dimensions in interpretability research, from approaches that provide "obviously" interpretable information to the studies of complex patterns. By applying the same categorization to interpretability in medical research, it is hoped that (1) clinicians and practitioners can subsequently approach these methods with caution, (2) insights into interpretability will be born with more considerations for medical practices, and (3) initiatives to push forward data-based, mathematically- and technically-grounded medical education are encouraged.

Machine Learning,Artificial Intelligence
Explainable Artificial Intelligence (XAI) for Deep Learning Based Medical Imaging Classification

Rawan Ghnemat,Sawsan Alodibat,Qasem Abu Al-Haija

DOI: https://doi.org/10.3390/jimaging9090177

2023-08-30

Abstract:Recently, deep learning has gained significant attention as a noteworthy division of artificial intelligence (AI) due to its high accuracy and versatile applications. However, one of the major challenges of AI is the need for more interpretability, commonly referred to as the black-box problem. In this study, we introduce an explainable AI model for medical image classification to enhance the interpretability of the decision-making process. Our approach is based on segmenting the images to provide a better understanding of how the AI model arrives at its results. We evaluated our model on five datasets, including the COVID-19 and Pneumonia Chest X-ray dataset, Chest X-ray (COVID-19 and Pneumonia), COVID-19 Image Dataset (COVID-19, Viral Pneumonia, Normal), and COVID-19 Radiography Database. We achieved testing and validation accuracy of 90.6% on a relatively small dataset of 6432 images. Our proposed model improved accuracy and reduced time complexity, making it more practical for medical diagnosis. Our approach offers a more interpretable and transparent AI model that can enhance the accuracy and efficiency of medical diagnosis.
Enhancing trustworthiness and reliability: advance explainable artificial intelligence framework for real world Sclerosis detection

Tanzila Saba,Muhammad Mujahid,Amjad Rehman,Faten S. Alamari,Noor Ayesha

DOI: https://doi.org/10.1088/1402-4896/ad72b0

2024-08-25

Physica Scripta

Abstract:In this era, Explainable Artificial Intelligence (XAI) is being employed in many health-related problems, but it faces challenges because most models produce results that are opaque and interpretable. The goal of explainable AI is to make machine learning, and deep learning models more understandable and accessible to people. Consequently, there is a pressing need for XAI models to enhance trust, given its increasing popularity in the field of medical artificial intelligence. This study explores the XAI nature of machine learning for disease prediction, with a particular focus on transparency and reliability of the results. The study examines the interpretability of artificial intelligence, focusing on issues such as bias, equality, and system reliability. The main theme is to minimize errors, disparities in human understanding, and use artificial intelligence in disease prediction to improve the outcomes for medical patients. The XAI methods were validated on Sclerosis predictions using two important models with fine-tuning their hyperparameters. The experiments demonstrated that the XAI methods outperformed the existing methods, achieving impressive results in terms of accuracy, recall, f1 score, precision, and AUC. The proposed approach achieved 98.53% accuracy using 75-25% hold-out splitting, and 98.14% accuracy using 10-fold validation. This semantic approach is superior to previous methods by showing the abundance of correct predictions and demonstrating its effectiveness in predicting multiple sclerosis in the real world.

physics, multidisciplinary
Self-eXplainable AI for Medical Image Analysis: A Survey and New Outlooks

Junlin Hou,Sicen Liu,Yequan Bie,Hongmei Wang,Andong Tan,Luyang Luo,Hao Chen

2024-10-03

Abstract:The increasing demand for transparent and reliable models, particularly in high-stakes decision-making areas such as medical image analysis, has led to the emergence of eXplainable Artificial Intelligence (XAI). Post-hoc XAI techniques, which aim to explain black-box models after training, have been controversial in recent works concerning their fidelity to the models' predictions. In contrast, Self-eXplainable AI (S-XAI) offers a compelling alternative by incorporating explainability directly into the training process of deep learning models. This approach allows models to generate inherent explanations that are closely aligned with their internal decision-making processes. Such enhanced transparency significantly supports the trustworthiness, robustness, and accountability of AI systems in real-world medical applications. To facilitate the development of S-XAI methods for medical image analysis, this survey presents an comprehensive review across various image modalities and clinical applications. It covers more than 200 papers from three key perspectives: 1) input explainability through the integration of explainable feature engineering and knowledge graph, 2) model explainability via attention-based learning, concept-based learning, and prototype-based learning, and 3) output explainability by providing counterfactual explanation and textual explanation. Additionally, this paper outlines the desired characteristics of explainability and existing evaluation methods for assessing explanation quality. Finally, it discusses the major challenges and future research directions in developing S-XAI for medical image analysis.

Computer Vision and Pattern Recognition
Explainable Artificial Intelligence (XAI) in Biomedicine: Making AI Decisions Trustworthy for Physicians and Patients

Jörn Lötsch,Dario Kringel,Alfred Ultsch

DOI: https://doi.org/10.3390/biomedinformatics2010001

2021-12-22

BioMedInformatics

Abstract:The use of artificial intelligence (AI) systems in biomedical and clinical settings can disrupt the traditional doctor–patient relationship, which is based on trust and transparency in medical advice and therapeutic decisions. When the diagnosis or selection of a therapy is no longer made solely by the physician, but to a significant extent by a machine using algorithms, decisions become nontransparent. Skill learning is the most common application of machine learning algorithms in clinical decision making. These are a class of very general algorithms (artificial neural networks, classifiers, etc.), which are tuned based on examples to optimize the classification of new, unseen cases. It is pointless to ask for an explanation for a decision. A detailed understanding of the mathematical details of an AI algorithm may be possible for experts in statistics or computer science. However, when it comes to the fate of human beings, this “developer’s explanation” is not sufficient. The concept of explainable AI (XAI) as a solution to this problem is attracting increasing scientific and regulatory interest. This review focuses on the requirement that XAIs must be able to explain in detail the decisions made by the AI to the experts in the field.
Analysis of Explainable Artificial Intelligence Methods on Medical Image Classification

Vinay Jogani,Joy Purohit,Ishaan Shivhare,Seema C Shrawne

DOI: https://doi.org/10.1109/ICONAT57137.2023.10080537

2022-12-10

Abstract:The use of deep learning in computer vision tasks such as image classification has led to a rapid increase in the performance of such systems. Due to this substantial increment in the utility of these systems, the use of artificial intelligence in many critical tasks has exploded. In the medical domain, medical image classification systems are being adopted due to their high accuracy and near parity with human physicians in many tasks. However, these artificial intelligence systems are extremely complex and are considered black boxes by scientists, due to the difficulty in interpreting what exactly led to the predictions made by these models. When these systems are being used to assist high-stakes decision-making, it is extremely important to be able to understand, verify and justify the conclusions reached by the model. The research techniques being used to gain insight into the black-box models are in the field of explainable artificial intelligence (XAI). In this paper, we evaluated three different XAI methods across two convolutional neural network models trained to classify lung cancer from histopathological images. We visualized the outputs and analyzed the performance of these methods, in order to better understand how to apply explainable artificial intelligence in the medical domain.

Image and Video Processing,Artificial Intelligence,Computer Vision and Pattern Recognition
Explainable AI for Medical Data: Current Methods, Limitations, and Future Directions

Imran Hossain,Ghada Zamzmi,Peter R. Mouton,Sirajus Salekin,Yu Sun,Dmitry Goldgof,Md Imran Hossain,Md Sirajus Salekin

DOI: https://doi.org/10.1145/3637487

IF: 16.6

2023-12-13

ACM Computing Surveys

Abstract:With the power of parallel processing, large datasets,and fast computational resources, deep neural networks (DNNs) have outperformed highly trained and experienced human experts in medical applications. However, the large global community of healthcare professionals, many of whom routinely face potentially life-or-death outcomes with complex medicolegal consequences, have yet to embrace this powerful technology. The major problem is that most current AI solutions function as a metaphorical black-box positioned between input data and output decisions without a rigorous explanation for their internal processes. With the goal of enhancing trust and improving acceptance of AI-based technology in clinical medicine, there is a large and growing effort to address this challenge using eXplainable AI (XAI), a set of techniques, strategies, and algorithms with an explicit focus on explaining the “hows and whys” of DNNs. Here, we provide a comprehensive review of the state-of-the-art XAI techniques concerning healthcare applications and discuss current challenges and future directions. We emphasize the strengths and limitations of each category, including image, tabular, and textual explanations, and explore a range of evaluation metrics for assessing the effectiveness of XAI solutions. Finally, we highlight promising opportunities for XAI research to enhance the acceptance of DNNs by the healthcare community.

computer science, theory & methods
The Role of Explainability in Assuring Safety of Machine Learning in Healthcare

Yan Jia,John McDermid,Tom Lawton,Ibrahim Habli

DOI: https://doi.org/10.48550/arXiv.2109.00520

2022-05-05

Abstract:Established approaches to assuring safety-critical systems and software are difficult to apply to systems employing ML where there is no clear, pre-defined specification against which to assess validity. This problem is exacerbated by the "opaque" nature of ML where the learnt model is not amenable to human scrutiny. Explainable AI (XAI) methods have been proposed to tackle this issue by producing human-interpretable representations of ML models which can help users to gain confidence and build trust in the ML system. However, little work explicitly investigates the role of explainability for safety assurance in the context of ML development. This paper identifies ways in which XAI methods can contribute to safety assurance of ML-based systems. It then uses a concrete ML-based clinical decision support system, concerning weaning of patients from mechanical ventilation, to demonstrate how XAI methods can be employed to produce evidence to support safety assurance. The results are also represented in a safety argument to show where, and in what way, XAI methods can contribute to a safety case. Overall, we conclude that XAI methods have a valuable role in safety assurance of ML-based systems in healthcare but that they are not sufficient in themselves to assure safety.

Machine Learning,Artificial Intelligence
Explainable AI applications in the Medical Domain: a systematic review

Nicoletta Prentzas,Antonis Kakas,Constantinos S. Pattichis

DOI: https://doi.org/10.48550/arXiv.2308.05411

2023-08-10

Abstract:Artificial Intelligence in Medicine has made significant progress with emerging applications in medical imaging, patient care, and other areas. While these applications have proven successful in retrospective studies, very few of them were applied in practice.The field of Medical AI faces various challenges, in terms of building user trust, complying with regulations, using data ethically.Explainable AI (XAI) aims to enable humans understand AI and trust its results. This paper presents a literature review on the recent developments of XAI solutions for medical decision support, based on a representative sample of 198 articles published in recent years. The systematic synthesis of the relevant articles resulted in several findings. (1) model-agnostic XAI techniques were mostly employed in these solutions, (2) deep learning models are utilized more than other types of machine learning models, (3) explainability was applied to promote trust, but very few works reported the physicians participation in the loop, (4) visual and interactive user interface is more useful in understanding the explanation and the recommendation of the system. More research is needed in collaboration between medical and AI experts, that could guide the development of suitable frameworks for the design, implementation, and evaluation of XAI solutions in medicine.

Artificial Intelligence,Machine Learning
eXplainable Artificial Intelligence on Medical Images: A Survey

Matteus Vargas Simão da Silva,Rodrigo Reis Arrais,Jhessica Victoria Santos da Silva,Felipe Souza Tânios,Mateus Antonio Chinelatto,Natalia Backhaus Pereira,Renata De Paris,Lucas Cesar Ferreira Domingos,Rodrigo Dória Villaça,Vitor Lopes Fabris,Nayara Rossi Brito da Silva,Ana Claudia Akemi Matsuki de Faria,Jose Victor Nogueira Alves da Silva,Fabiana Cristina Queiroz de Oliveira Marucci,Francisco Alves de Souza Neto,Danilo Xavier Silva,Vitor Yukio Kondo,Claudio Filipi Gonçalves dos Santos

2023-05-12

Abstract:Over the last few years, the number of works about deep learning applied to the medical field has increased enormously. The necessity of a rigorous assessment of these models is required to explain these results to all people involved in medical exams. A recent field in the machine learning area is explainable artificial intelligence, also known as XAI, which targets to explain the results of such black box models to permit the desired assessment. This survey analyses several recent studies in the XAI field applied to medical diagnosis research, allowing some explainability of the machine learning results in several different diseases, such as cancers and COVID-19.

Machine Learning,Artificial Intelligence,Computers and Society,Image and Video Processing
What do we need to build explainable AI systems for the medical domain?

Andreas Holzinger,Chris Biemann,Constantinos S. Pattichis,Douglas B. Kell

DOI: https://doi.org/10.48550/arXiv.1712.09923

IF: 14.4

2017-12-28

Artificial Intelligence

Abstract:Artificial intelligence (AI) generally and machine learning (ML) specifically demonstrate impressive practical success in many different application domains, e.g. in autonomous driving, speech recognition, or recommender systems. Deep learning approaches, trained on extremely large data sets or using reinforcement learning methods have even exceeded human performance in visual tasks, particularly on playing games such as Atari, or mastering the game of Go. Even in the medical domain there are remarkable results. The central problem of such models is that they are regarded as black-box models and even if we understand the underlying mathematical principles, they lack an explicit declarative knowledge representation, hence have difficulty in generating the underlying explanatory structures. This calls for systems enabling to make decisions transparent, understandable and explainable. A huge motivation for our approach are rising legal and privacy aspects. The new European General Data Protection Regulation entering into force on May 25th 2018, will make black-box approaches difficult to use in business. This does not imply a ban on automatic learning approaches or an obligation to explain everything all the time, however, there must be a possibility to make the results re-traceable on demand. In this paper we outline some of our research topics in the context of the relatively new area of explainable-AI with a focus on the application in medicine, which is a very special domain. This is due to the fact that medical professionals are working mostly with distributed heterogeneous and complex sources of data. In this paper we concentrate on three sources: images, *omics data and text. We argue that research in explainable-AI would generally help to facilitate the implementation of AI/ML in the medical domain, and specifically help to facilitate transparency and trust.
Improving trust and confidence in medical skin lesion diagnosis through explainable deep learning

Carlo Metta,Andrea Beretta,Riccardo Guidotti,Yuan Yin,Patrick Gallinari,Salvatore Rinzivillo,Fosca Giannotti

DOI: https://doi.org/10.1007/s41060-023-00401-z

2023-06-21

International Journal of Data Science and Analytics

Abstract:Abstract A key issue in critical contexts such as medical diagnosis is the interpretability of the deep learning models adopted in decision-making systems. Research in eXplainable Artificial Intelligence (XAI) is trying to solve this issue. However, often XAI approaches are only tested on generalist classifier and do not represent realistic problems such as those of medical diagnosis. In this paper, we aim at improving the trust and confidence of users towards automatic AI decision systems in the field of medical skin lesion diagnosis by customizing an existing XAI approach for explaining an AI model able to recognize different types of skin lesions. The explanation is generated through the use of synthetic exemplar and counter-exemplar images of skin lesions and our contribution offers the practitioner a way to highlight the crucial traits responsible for the classification decision. A validation survey with domain experts, beginners, and unskilled people shows that the use of explanations improves trust and confidence in the automatic decision system. Also, an analysis of the latent space adopted by the explainer unveils that some of the most frequent skin lesion classes are distinctly separated. This phenomenon may stem from the intrinsic characteristics of each class and may help resolve common misclassifications made by human experts.
Explainable Artificial Intelligence and Cardiac Imaging: Toward More Interpretable Models.

A. Lee,P. Gkontra,Z. Raisi-Estabragh,Ahmed M. A. Salih,Ilaria Boscolo Galazzo,Steffen E. Petersen,K. Lekadir

DOI: https://doi.org/10.1161/CIRCIMAGING.122.014519

2023-04-12

Circulation Cardiovascular Imaging

Abstract:Artificial intelligence applications have shown success in different medical and health care domains, and cardiac imaging is no exception. However, some machine learning models, especially deep learning, are considered black box as they do not provide an explanation or rationale for model outcomes. Complexity and vagueness in these models necessitate a transition to explainable artificial intelligence (XAI) methods to ensure that model results are both transparent and understandable to end users. In cardiac imaging studies, there are a limited number of papers that use XAI methodologies. This article provides a comprehensive literature review of state-of-the-art works using XAI methods for cardiac imaging. Moreover, it provides simple and comprehensive guidelines on XAI. Finally, open issues and directions for XAI in cardiac imaging are discussed.

Medicine,Computer Science
The Limits of Perception: Analyzing Inconsistencies in Saliency Maps in XAI

Anna Stubbin,Thompson Chyrikov,Jim Zhao,Christina Chajo

2024-03-23

Abstract:Explainable artificial intelligence (XAI) plays an indispensable role in demystifying the decision-making processes of AI, especially within the healthcare industry. Clinicians rely heavily on detailed reasoning when making a diagnosis, often CT scans for specific features that distinguish between benign and malignant lesions. A comprehensive diagnostic approach includes an evaluation of imaging results, patient observations, and clinical tests. The surge in deploying deep learning models as support systems in medical diagnostics has been significant, offering advances that traditional methods could not. However, the complexity and opacity of these models present a double-edged sword. As they operate as "black boxes," with their reasoning obscured and inaccessible, there's an increased risk of misdiagnosis, which can lead to patient harm. Hence, there is a pressing need to cultivate transparency within AI systems, ensuring that the rationale behind an AI's diagnostic recommendations is clear and understandable to medical practitioners. This shift towards transparency is not just beneficial -- it's a critical step towards responsible AI integration in healthcare, ensuring that AI aids rather than hinders medical professionals in their crucial work.

Computer Vision and Pattern Recognition
A Survey on Medical Explainable AI (XAI): Recent Progress, Explainability Approach, Human Interaction and Scoring System

Ruey-Kai Sheu,Mayuresh Sunil Pardeshi

DOI: https://doi.org/10.3390/s22208068

IF: 3.9

2022-10-22

Sensors

Abstract:The emerging field of eXplainable AI (XAI) in the medical domain is considered to be of utmost importance. Meanwhile, incorporating explanations in the medical domain with respect to legal and ethical AI is necessary to understand detailed decisions, results, and current status of the patient's conditions. Successively, we will be presenting a detailed survey for the medical XAI with the model enhancements, evaluation methods, significant overview of case studies with open box architecture, medical open datasets, and future improvements. Potential differences in AI and XAI methods are provided with the recent XAI methods stated as (i) local and global methods for preprocessing, (ii) knowledge base and distillation algorithms, and (iii) interpretable machine learning. XAI characteristics details with future healthcare explainability is included prominently, whereas the pre-requisite provides insights for the brainstorming sessions before beginning a medical XAI project. Practical case study determines the recent XAI progress leading to the advance developments within the medical field. Ultimately, this survey proposes critical ideas surrounding a user-in-the-loop approach, with an emphasis on human–machine collaboration, to better produce explainable solutions. The surrounding details of the XAI feedback system for human rating-based evaluation provides intelligible insights into a constructive method to produce human enforced explanation feedback. For a long time, XAI limitations of the ratings, scores and grading are present. Therefore, a novel XAI recommendation system and XAI scoring system are designed and approached from this work. Additionally, this paper encourages the importance of implementing explainable solutions into the high impact medical field.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation

TICAS assessment of cells from atypical hyperplasia of the endometrium.

An Explainable AI Framework for Artificial Intelligence of Medical Things

Explainable Machine Learning-Based Security and Privacy Protection Framework for Internet of Medical Things Systems

Unveiling the black box: A systematic review of Explainable Artificial Intelligence in medical image analysis

Explainable Artificial Intelligence for Medical Applications: A Review

Transparency and Trust in Human-AI-Interaction: The Role of Model-Agnostic Explanations in Computer Vision-Based Decision Support

A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI

Explainable Artificial Intelligence (XAI) for Deep Learning Based Medical Imaging Classification

Enhancing trustworthiness and reliability: advance explainable artificial intelligence framework for real world Sclerosis detection

Self-eXplainable AI for Medical Image Analysis: A Survey and New Outlooks

Explainable Artificial Intelligence (XAI) in Biomedicine: Making AI Decisions Trustworthy for Physicians and Patients

Analysis of Explainable Artificial Intelligence Methods on Medical Image Classification

Explainable AI for Medical Data: Current Methods, Limitations, and Future Directions

The Role of Explainability in Assuring Safety of Machine Learning in Healthcare

Explainable AI applications in the Medical Domain: a systematic review

eXplainable Artificial Intelligence on Medical Images: A Survey

What do we need to build explainable AI systems for the medical domain?

Improving trust and confidence in medical skin lesion diagnosis through explainable deep learning

Explainable Artificial Intelligence and Cardiac Imaging: Toward More Interpretable Models.

The Limits of Perception: Analyzing Inconsistencies in Saliency Maps in XAI

A Survey on Medical Explainable AI (XAI): Recent Progress, Explainability Approach, Human Interaction and Scoring System