Antibodies against Actinobacillus pleuropneumoniae serotype 2 in mucosal secretions and sera of infected pigs as demonstrated by an enzyme-linked immunosorbent assay.

Loftager Mk,L. Eriksen,R. Nielsen

DOI: https://doi.org/10.1016/0034-5288(93)90011-4

IF: 2.554

Research in Veterinary Science

Abstract:

What problem does this paper attempt to address?

Explainable Artificial Intelligence for Medical Applications: A Review

Qiyang Sun,Alican Akman,Björn W. Schuller

2024-11-15

Abstract:The continuous development of artificial intelligence (AI) theory has propelled this field to unprecedented heights, owing to the relentless efforts of scholars and researchers. In the medical realm, AI takes a pivotal role, leveraging robust machine learning (ML) algorithms. AI technology in medical imaging aids physicians in X-ray, computed tomography (CT) scans, and magnetic resonance imaging (MRI) diagnoses, conducts pattern recognition and disease prediction based on acoustic data, delivers prognoses on disease types and developmental trends for patients, and employs intelligent health management wearable devices with human-computer interaction technology to name but a few. While these well-established applications have significantly assisted in medical field diagnoses, clinical decision-making, and management, collaboration between the medical and AI sectors faces an urgent challenge: How to substantiate the reliability of decision-making? The underlying issue stems from the conflict between the demand for accountability and result transparency in medical scenarios and the black-box model traits of AI. This article reviews recent research grounded in explainable artificial intelligence (XAI), with an emphasis on medical practices within the visual, audio, and multimodal perspectives. We endeavour to categorise and synthesise these practices, aiming to provide support and guidance for future researchers and healthcare professionals.

Machine Learning,Computer Vision and Pattern Recognition
Should AI models be explainable to clinicians?

Gwénolé Abgrall,Andre L. Holder,Zaineb Chelly Dagdia,Karine Zeitouni,Xavier Monnet

DOI: https://doi.org/10.1186/s13054-024-05005-y

IF: 15.1

2024-09-14

Critical Care

Abstract:In the high-stakes realm of critical care, where daily decisions are crucial and clear communication is paramount, comprehending the rationale behind Artificial Intelligence (AI)-driven decisions appears essential. While AI has the potential to improve decision-making, its complexity can hinder comprehension and adherence to its recommendations. "Explainable AI" (XAI) aims to bridge this gap, enhancing confidence among patients and doctors. It also helps to meet regulatory transparency requirements, offers actionable insights, and promotes fairness and safety. Yet, defining explainability and standardising assessments are ongoing challenges and balancing performance and explainability can be needed, even if XAI is a growing field.

critical care medicine
Explainable AI for clinical risk prediction: a survey of concepts, methods, and modalities

Munib Mesinovic,Peter Watkinson,Tingting Zhu

DOI: https://doi.org/10.48550/arXiv.2308.08407

2023-08-16

Abstract:Recent advancements in AI applications to healthcare have shown incredible promise in surpassing human performance in diagnosis and disease prognosis. With the increasing complexity of AI models, however, concerns regarding their opacity, potential biases, and the need for interpretability. To ensure trust and reliability in AI systems, especially in clinical risk prediction models, explainability becomes crucial. Explainability is usually referred to as an AI system's ability to provide a robust interpretation of its decision-making logic or the decisions themselves to human stakeholders. In clinical risk prediction, other aspects of explainability like fairness, bias, trust, and transparency also represent important concepts beyond just interpretability. In this review, we address the relationship between these concepts as they are often used together or interchangeably. This review also discusses recent progress in developing explainable models for clinical risk prediction, highlighting the importance of quantitative and clinical evaluation and validation across multiple common modalities in clinical practice. It emphasizes the need for external validation and the combination of diverse interpretability methods to enhance trust and fairness. Adopting rigorous testing, such as using synthetic datasets with known generative factors, can further improve the reliability of explainability methods. Open access and code-sharing resources are essential for transparency and reproducibility, enabling the growth and trustworthiness of explainable research. While challenges exist, an end-to-end approach to explainability in clinical risk prediction, incorporating stakeholders from clinicians to developers, is essential for success.

Machine Learning,Artificial Intelligence,Computers and Society
To explain or not to explain?-Artificial intelligence explainability in clinical decision support systems

Julia Amann,Dennis Vetter,Stig Nikolaj Blomberg,Helle Collatz Christensen,Megan Coffee,Sara Gerke,Thomas K Gilbert,Thilo Hagendorff,Sune Holm,Michelle Livne,Andy Spezzatti,Inga Strümke,Roberto V Zicari,Vince Istvan Madai,Z-Inspection initiative

DOI: https://doi.org/10.1371/journal.pdig.0000016

2022-02-17

Abstract:Explainability for artificial intelligence (AI) in medicine is a hotly debated topic. Our paper presents a review of the key arguments in favor and against explainability for AI-powered Clinical Decision Support System (CDSS) applied to a concrete use case, namely an AI-powered CDSS currently used in the emergency call setting to identify patients with life-threatening cardiac arrest. More specifically, we performed a normative analysis using socio-technical scenarios to provide a nuanced account of the role of explainability for CDSSs for the concrete use case, allowing for abstractions to a more general level. Our analysis focused on three layers: technical considerations, human factors, and the designated system role in decision-making. Our findings suggest that whether explainability can provide added value to CDSS depends on several key questions: technical feasibility, the level of validation in case of explainable algorithms, the characteristics of the context in which the system is implemented, the designated role in the decision-making process, and the key user group(s). Thus, each CDSS will require an individualized assessment of explainability needs and we provide an example of how such an assessment could look like in practice.
Explainable Artificial Intelligence for Predictive Modeling in Healthcare

Christopher C. Yang

DOI: https://doi.org/10.1007/s41666-022-00114-1

2022-02-11

Journal of Healthcare Informatics Research

Abstract:The principle behind artificial intelligence is mimicking human intelligence in the way that it can perform tasks, recognize patterns, or predict outcomes through learning from the acquired data of various sources. Artificial intelligence and machine learning algorithms have been widely used in autonomous driving, recommender systems in electronic commerce and social media, fintech, natural language understanding, and question answering systems. Artificial intelligence is also gradually changing the landscape of healthcare research (Yu et al. in Biomed Eng 2:719–731, 25). The rule-based approach that relied on the curation of medical knowledge and the construction of robust decision rules had drawn significant attention in diagnosing diseases and clinical decision support since half a century ago. In recent years, machine learning algorithms such as deep learning that can account for complex interactions between features is shown to be promising in predictive modeling in healthcare (Deo in Circulation 132:1920–1930, 26). Although many of these artificial intelligence and machine learning algorithms can achieve remarkably high performance, it is often difficult to be completely adopted in practical clinical environments due to the lack of explainability in some of these algorithms. Explainable artificial intelligence (XAI) is emerging to assist in the communication of internal decisions, behavior, and actions to health care professionals. Through explaining the prediction outcomes, XAI gains the trust of the clinicians as they may learn how to apply the predictive modeling in practical situations instead of blindly following the predictions. There are still many scenarios to explore how to make XAI effective in clinical settings due to the complexity of medical knowledge.
The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies

Aniek F. Markus,Jan A. Kors,Peter R. Rijnbeek

DOI: https://doi.org/10.1016/j.jbi.2020.103655

2021-01-05

Abstract:Artificial intelligence (AI) has huge potential to improve the health and well-being of people, but adoption in clinical practice is still limited. Lack of transparency is identified as one of the main barriers to implementation, as clinicians should be confident the AI system can be trusted. Explainable AI has the potential to overcome this issue and can be a step towards trustworthy AI. In this paper we review the recent literature to provide guidance to researchers and practitioners on the design of explainable AI systems for the health-care domain and contribute to formalization of the field of explainable AI. We argue the reason to demand explainability determines what should be explained as this determines the relative importance of the properties of explainability (i.e. interpretability and fidelity). Based on this, we propose a framework to guide the choice between classes of explainable AI methods (explainable modelling versus post-hoc explanation; model-based, attribution-based, or example-based explanations; global and local explanations). Furthermore, we find that quantitative evaluation metrics, which are important for objective standardized evaluation, are still lacking for some properties (e.g. clarity) and types of explanations (e.g. example-based methods). We conclude that explainable modelling can contribute to trustworthy AI, but the benefits of explainability still need to be proven in practice and complementary measures might be needed to create trustworthy AI in health care (e.g. reporting data quality, performing extensive (external) validation, and regulation).

Artificial Intelligence,Machine Learning
What do we need to build explainable AI systems for the medical domain?

Andreas Holzinger,Chris Biemann,Constantinos S. Pattichis,Douglas B. Kell

DOI: https://doi.org/10.48550/arXiv.1712.09923

IF: 14.4

2017-12-28

Artificial Intelligence

Abstract:Artificial intelligence (AI) generally and machine learning (ML) specifically demonstrate impressive practical success in many different application domains, e.g. in autonomous driving, speech recognition, or recommender systems. Deep learning approaches, trained on extremely large data sets or using reinforcement learning methods have even exceeded human performance in visual tasks, particularly on playing games such as Atari, or mastering the game of Go. Even in the medical domain there are remarkable results. The central problem of such models is that they are regarded as black-box models and even if we understand the underlying mathematical principles, they lack an explicit declarative knowledge representation, hence have difficulty in generating the underlying explanatory structures. This calls for systems enabling to make decisions transparent, understandable and explainable. A huge motivation for our approach are rising legal and privacy aspects. The new European General Data Protection Regulation entering into force on May 25th 2018, will make black-box approaches difficult to use in business. This does not imply a ban on automatic learning approaches or an obligation to explain everything all the time, however, there must be a possibility to make the results re-traceable on demand. In this paper we outline some of our research topics in the context of the relatively new area of explainable-AI with a focus on the application in medicine, which is a very special domain. This is due to the fact that medical professionals are working mostly with distributed heterogeneous and complex sources of data. In this paper we concentrate on three sources: images, *omics data and text. We argue that research in explainable-AI would generally help to facilitate the implementation of AI/ML in the medical domain, and specifically help to facilitate transparency and trust.
A manifesto on explainability for artificial intelligence in medicine

Carlo Combi,Beatrice Amico,Riccardo Bellazzi,Andreas Holzinger,Marinka Zitnik,Jason H. Moore,John H. Holmes

DOI: https://doi.org/10.1016/j.artmed.2022.102423

IF: 7.011

2022-11-01

Artificial Intelligence in Medicine

Abstract:The rapid increase of interest in, and use of, artificial intelligence (AI) in computer applications has raised a parallel concern about its ability (or lack thereof) to provide understandable, or explainable, output to users. This concern is especially legitimate in biomedical contexts, where patient safety is of paramount importance. This position paper brings together seven researchers working in the field with different roles and perspectives, to explore in depth the concept of explainable AI, or XAI, offering a functional definition and conceptual framework or model that can be used when considering XAI. This is followed by a series of desiderata for attaining explainability in AI, each of which touches upon a key domain in biomedicine.

engineering, biomedical,computer science, artificial intelligence,medical informatics
Explainable AI for Medical Data: Current Methods, Limitations, and Future Directions

Imran Hossain,Ghada Zamzmi,Peter R. Mouton,Sirajus Salekin,Yu Sun,Dmitry Goldgof,Md Imran Hossain,Md Sirajus Salekin

DOI: https://doi.org/10.1145/3637487

IF: 16.6

2023-12-13

ACM Computing Surveys

Abstract:With the power of parallel processing, large datasets,and fast computational resources, deep neural networks (DNNs) have outperformed highly trained and experienced human experts in medical applications. However, the large global community of healthcare professionals, many of whom routinely face potentially life-or-death outcomes with complex medicolegal consequences, have yet to embrace this powerful technology. The major problem is that most current AI solutions function as a metaphorical black-box positioned between input data and output decisions without a rigorous explanation for their internal processes. With the goal of enhancing trust and improving acceptance of AI-based technology in clinical medicine, there is a large and growing effort to address this challenge using eXplainable AI (XAI), a set of techniques, strategies, and algorithms with an explicit focus on explaining the “hows and whys” of DNNs. Here, we provide a comprehensive review of the state-of-the-art XAI techniques concerning healthcare applications and discuss current challenges and future directions. We emphasize the strengths and limitations of each category, including image, tabular, and textual explanations, and explore a range of evaluation metrics for assessing the effectiveness of XAI solutions. Finally, we highlight promising opportunities for XAI research to enhance the acceptance of DNNs by the healthcare community.

computer science, theory & methods
Explainable AI applications in the Medical Domain: a systematic review

Nicoletta Prentzas,Antonis Kakas,Constantinos S. Pattichis

DOI: https://doi.org/10.48550/arXiv.2308.05411

2023-08-10

Abstract:Artificial Intelligence in Medicine has made significant progress with emerging applications in medical imaging, patient care, and other areas. While these applications have proven successful in retrospective studies, very few of them were applied in practice.The field of Medical AI faces various challenges, in terms of building user trust, complying with regulations, using data ethically.Explainable AI (XAI) aims to enable humans understand AI and trust its results. This paper presents a literature review on the recent developments of XAI solutions for medical decision support, based on a representative sample of 198 articles published in recent years. The systematic synthesis of the relevant articles resulted in several findings. (1) model-agnostic XAI techniques were mostly employed in these solutions, (2) deep learning models are utilized more than other types of machine learning models, (3) explainability was applied to promote trust, but very few works reported the physicians participation in the loop, (4) visual and interactive user interface is more useful in understanding the explanation and the recommendation of the system. More research is needed in collaboration between medical and AI experts, that could guide the development of suitable frameworks for the design, implementation, and evaluation of XAI solutions in medicine.

Artificial Intelligence,Machine Learning
Explainable Artificial Intelligence (XAI) in Biomedicine: Making AI Decisions Trustworthy for Physicians and Patients

Jörn Lötsch,Dario Kringel,Alfred Ultsch

DOI: https://doi.org/10.3390/biomedinformatics2010001

2021-12-22

BioMedInformatics

Abstract:The use of artificial intelligence (AI) systems in biomedical and clinical settings can disrupt the traditional doctor–patient relationship, which is based on trust and transparency in medical advice and therapeutic decisions. When the diagnosis or selection of a therapy is no longer made solely by the physician, but to a significant extent by a machine using algorithms, decisions become nontransparent. Skill learning is the most common application of machine learning algorithms in clinical decision making. These are a class of very general algorithms (artificial neural networks, classifiers, etc.), which are tuned based on examples to optimize the classification of new, unseen cases. It is pointless to ask for an explanation for a decision. A detailed understanding of the mathematical details of an AI algorithm may be possible for experts in statistics or computer science. However, when it comes to the fate of human beings, this “developer’s explanation” is not sufficient. The concept of explainable AI (XAI) as a solution to this problem is attracting increasing scientific and regulatory interest. This review focuses on the requirement that XAIs must be able to explain in detail the decisions made by the AI to the experts in the field.
Why we do need Explainable AI for Healthcare

Giovanni Cinà,Tabea Röber,Rob Goedhart,Ilker Birbil

DOI: https://doi.org/10.48550/arXiv.2206.15363

2022-06-30

Abstract:The recent spike in certified Artificial Intelligence (AI) tools for healthcare has renewed the debate around adoption of this technology. One thread of such debate concerns Explainable AI and its promise to render AI devices more transparent and trustworthy. A few voices active in the medical AI space have expressed concerns on the reliability of Explainable AI techniques, questioning their use and inclusion in guidelines and standards. Revisiting such criticisms, this article offers a balanced and comprehensive perspective on the utility of Explainable AI, focusing on the specificity of clinical applications of AI and placing them in the context of healthcare interventions. Against its detractors and despite valid concerns, we argue that the Explainable AI research program is still central to human-machine interaction and ultimately our main tool against loss of control, a danger that cannot be prevented by rigorous clinical validation alone.

Human-Computer Interaction,Artificial Intelligence,Machine Learning
A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?

Subrato Bharati,M. Rubaiyat Hossain Mondal,Prajoy Podder

DOI: https://doi.org/10.1109/TAI.2023.3266418

2023-04-11

Abstract:Artificial intelligence (AI) models are increasingly finding applications in the field of medicine. Concerns have been raised about the explainability of the decisions that are made by these AI models. In this article, we give a systematic analysis of explainable artificial intelligence (XAI), with a primary focus on models that are currently being used in the field of healthcare. The literature search is conducted following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) standards for relevant work published from 1 January 2012 to 02 February 2022. The review analyzes the prevailing trends in XAI and lays out the major directions in which research is headed. We investigate the why, how, and when of the uses of these XAI models and their implications. We present a comprehensive examination of XAI methodologies as well as an explanation of how a trustworthy AI can be derived from describing AI models for healthcare fields. The discussion of this work will contribute to the formalization of the XAI field.

Machine Learning,Artificial Intelligence
A Brief Review of Explainable Artificial Intelligence in Healthcare

Zahra Sadeghi,Roohallah Alizadehsani,Mehmet Akif Cifci,Samina Kausar,Rizwan Rehman,Priyakshi Mahanta,Pranjal Kumar Bora,Ammar Almasri,Rami S. Alkhawaldeh,Sadiq Hussain,Bilal Alatas,Afshin Shoeibi,Hossein Moosaei,Milan Hladik,Saeid Nahavandi,Panos M. Pardalos

2023-04-04

Abstract:XAI refers to the techniques and methods for building AI applications which assist end users to interpret output and predictions of AI models. Black box AI applications in high-stakes decision-making situations, such as medical domain have increased the demand for transparency and explainability since wrong predictions may have severe consequences. Model explainability and interpretability are vital successful deployment of AI models in healthcare practices. AI applications' underlying reasoning needs to be transparent to clinicians in order to gain their trust. This paper presents a systematic review of XAI aspects and challenges in the healthcare domain. The primary goals of this study are to review various XAI methods, their challenges, and related machine learning models in healthcare. The methods are discussed under six categories: Features-oriented methods, global methods, concept models, surrogate models, local pixel-based methods, and human-centric methods. Most importantly, the paper explores XAI role in healthcare problems to clarify its necessity in safety-critical applications. The paper intends to establish a comprehensive understanding of XAI-related applications in the healthcare field by reviewing the related experimental results. To facilitate future research for filling research gaps, the importance of XAI models from different viewpoints and their limitations are investigated.

Artificial Intelligence
Explainable Artificial Intelligence (XAI): Concepts and Challenges in Healthcare

Tim Hulsen

DOI: https://doi.org/10.3390/ai4030034

2023-08-10

AI

Abstract:Artificial Intelligence (AI) describes computer systems able to perform tasks that normally require human intelligence, such as visual perception, speech recognition, decision-making, and language translation. Examples of AI techniques are machine learning, neural networks, and deep learning. AI can be applied in many different areas, such as econometrics, biometry, e-commerce, and the automotive industry. In recent years, AI has found its way into healthcare as well, helping doctors make better decisions (“clinical decision support”), localizing tumors in magnetic resonance images, reading and analyzing reports written by radiologists and pathologists, and much more. However, AI has one big risk: it can be perceived as a “black box”, limiting trust in its reliability, which is a very big issue in an area in which a decision can mean life or death. As a result, the term Explainable Artificial Intelligence (XAI) has been gaining momentum. XAI tries to ensure that AI algorithms (and the resulting decisions) can be understood by humans. In this narrative review, we will have a look at some central concepts in XAI, describe several challenges around XAI in healthcare, and discuss whether it can really help healthcare to advance, for example, by increasing understanding and trust. Finally, alternatives to increase trust in AI are discussed, as well as future research possibilities in the area of XAI.
Explainable AI: Definition and attributes of a good explanation for health AI

Evangelia Kyrimi,Scott McLachlan,Jared M Wohlgemut,Zane B Perkins,David A. Lagnado,William Marsh,ExAIDSS Expert Group

2024-09-10

Abstract:Proposals of artificial intelligence (AI) solutions based on increasingly complex and accurate predictive models are becoming ubiquitous across many disciplines. As the complexity of these models grows, transparency and users' understanding often diminish. This suggests that accurate prediction alone is insufficient for making an AI-based solution truly useful. In the development of healthcare systems, this introduces new issues related to accountability and safety. Understanding how and why an AI system makes a recommendation may require complex explanations of its inner workings and reasoning processes. Although research on explainable AI (XAI) has significantly increased in recent years and there is high demand for XAI in medicine, defining what constitutes a good explanation remains ad hoc, and providing adequate explanations continues to be challenging. To fully realize the potential of AI, it is critical to address two fundamental questions about explanations for safety-critical AI applications, such as health-AI: (1) What is an explanation in health-AI? and (2) What are the attributes of a good explanation in health-AI? In this study, we examined published literature and gathered expert opinions through a two-round Delphi study. The research outputs include (1) a definition of what constitutes an explanation in health-AI and (2) a comprehensive list of attributes that characterize a good explanation in health-AI.

Computers and Society,Artificial Intelligence
A review of explainable and interpretable AI with applications in COVID‐19 imaging

Jordan D. Fuhrman,Naveena Gorre,Qiyuan Hu,Hui Li,Issam El Naqa,Maryellen L. Giger

DOI: https://doi.org/10.1002/mp.15359

IF: 4.506

2021-12-07

Medical Physics

Abstract:The development of medical imaging artificial intelligence (AI) systems for evaluating COVID-19 patients has demonstrated potential for improving clinical decision making and assessing patient outcomes during the recent COVID-19 pandemic. These have been applied to many medical imaging tasks, including disease diagnosis and patient prognosis, as well as augmented other clinical measurements to better inform treatment decisions. Because these systems are used in life-or-death decisions, clinical implementation relies on user trust in the AI output. This has caused many developers to utilize explainability techniques in an attempt to help a user understand when an AI algorithm is likely to succeed as well as which cases may be problematic for automatic assessment, thus increasing the potential for rapid clinical translation. AI application to COVID-19 has been marred with controversy recently. This review discusses several aspects of explainable and interpretable AI as it pertains to the evaluation of COVID-19 disease and it can restore trust in AI application to this disease. This includes the identification of common tasks that are relevant to explainable medical imaging AI, an overview of several modern approaches for producing explainable output as appropriate for a given imaging scenario, a discussion of how to evaluate explainable AI, and recommendations for best practices in explainable/interpretable AI implementation. This review will allow developers of AI systems for COVID-19 to quickly understand the basics of several explainable AI techniques and assist in the selection of an approach that is both appropriate and effective for a given scenario.

radiology, nuclear medicine & medical imaging
The Explanation Necessity for Healthcare AI

Michail Mamalakis,Héloïse de Vareilles,Graham Murray,Pietro Lio,John Suckling

2024-06-01

Abstract:Explainability is often critical to the acceptable implementation of artificial intelligence (AI). Nowhere is this more important than healthcare where decision-making directly impacts patients and trust in AI systems is essential. This trust is often built on the explanations and interpretations the AI provides. Despite significant advancements in AI interpretability, there remains the need for clear guidelines on when and to what extent explanations are necessary in the medical context. We propose a novel categorization system with four distinct classes of explanation necessity, guiding the level of explanation required: patient or sample (local) level, cohort or dataset (global) level, or both levels. We introduce a mathematical formulation that distinguishes these categories and offers a practical framework for researchers to determine the necessity and depth of explanations required in medical AI applications. Three key factors are considered: the robustness of the evaluation protocol, the variability of expert observations, and the representation dimensionality of the application. In this perspective, we address the question: When does an AI medical application need to be explained, and at what level of detail?

Artificial Intelligence
The role of explainable Artificial Intelligence in high-stakes decision-making systems: a systematic review

DOI: https://doi.org/10.1007/s12652-023-04594-w

IF: 3.662

2023-04-04

Journal of Ambient Intelligence and Humanized Computing

Abstract:A high-stakes event is an extreme risk with a low probability of occurring, but severe consequences (e.g., life-threatening conditions or economic collapse). The accompanying lack of information is a source of high-stress pressure and anxiety for emergency medical services authorities. Deciding on the best proactive plan and action in this environment is a complicated process, which calls for intelligent agents to automatically produce knowledge in the manner of human-like intelligence. Research in high-stakes decision-making systems has increasingly focused on eXplainable Artificial Intelligence (XAI), but recent developments in prediction systems give little prominence to explanations based on human-like intelligence. This work investigates XAI based on cause-and-effect interpretations for supporting high-stakes decisions. We review recent applications in the first aid and medical emergency fields based on three perspectives: available data, desirable knowledge, and the use of intelligence. We identify the limitations of recent AI, and discuss the potential of XAI for dealing with such limitations. We propose an architecture for high-stakes decision-making driven by XAI, and highlight likely future trends and directions.

computer science, information systems,telecommunications, artificial intelligence
Evaluation of Popular XAI Applied to Clinical Prediction Models: Can They be Trusted?

Aida Brankovic,David Cook,Jessica Rahman,Wenjie Huang,Sankalp Khanna

2023-06-21

Abstract:The absence of transparency and explainability hinders the clinical adoption of Machine learning (ML) algorithms. Although various methods of explainable artificial intelligence (XAI) have been suggested, there is a lack of literature that delves into their practicality and assesses them based on criteria that could foster trust in clinical environments. To address this gap this study evaluates two popular XAI methods used for explaining predictive models in the healthcare context in terms of whether they (i) generate domain-appropriate representation, i.e. coherent with respect to the application task, (ii) impact clinical workflow and (iii) are consistent. To that end, explanations generated at the cohort and patient levels were analysed. The paper reports the first benchmarking of the XAI methods applied to risk prediction models obtained by evaluating the concordance between generated explanations and the trigger of a future clinical deterioration episode recorded by the data collection system. We carried out an analysis using two Electronic Medical Records (EMR) datasets sourced from Australian major hospitals. The findings underscore the limitations of state-of-the-art XAI methods in the clinical context and their potential benefits. We discuss these limitations and contribute to the theoretical development of trustworthy XAI solutions where clinical decision support guides the choice of intervention by suggesting the pattern or drivers for clinical deterioration in the future.

Machine Learning,Computers and Society

Antibodies against Actinobacillus pleuropneumoniae serotype 2 in mucosal secretions and sera of infected pigs as demonstrated by an enzyme-linked immunosorbent assay.

Explainable Artificial Intelligence for Medical Applications: A Review

Should AI models be explainable to clinicians?

Explainable AI for clinical risk prediction: a survey of concepts, methods, and modalities

To explain or not to explain?-Artificial intelligence explainability in clinical decision support systems

Explainable Artificial Intelligence for Predictive Modeling in Healthcare

The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies

What do we need to build explainable AI systems for the medical domain?

A manifesto on explainability for artificial intelligence in medicine

Explainable AI for Medical Data: Current Methods, Limitations, and Future Directions

Explainable AI applications in the Medical Domain: a systematic review

Explainable Artificial Intelligence (XAI) in Biomedicine: Making AI Decisions Trustworthy for Physicians and Patients

Why we do need Explainable AI for Healthcare

A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?

A Brief Review of Explainable Artificial Intelligence in Healthcare

Explainable Artificial Intelligence (XAI): Concepts and Challenges in Healthcare

Explainable AI: Definition and attributes of a good explanation for health AI

A review of explainable and interpretable AI with applications in COVID‐19 imaging

The Explanation Necessity for Healthcare AI

The role of explainable Artificial Intelligence in high-stakes decision-making systems: a systematic review

Evaluation of Popular XAI Applied to Clinical Prediction Models: Can They be Trusted?