IEEE Signal Processing Magazine Special Issue on Explainability in Data Science: Interpretability, Reproducibility, and Replicability

DOI: https://doi.org/10.1109/msp.2021.3049658

IF: 15.204

2021-03-01

IEEE Signal Processing Magazine

Abstract:Prospective authors are requested to submit new, unpublished manuscripts for inclusion in the upcoming event described in this call for papers.

What problem does this paper attempt to address?

IEEE Signal Processing Magazine Special Issue: Innovation Starts with Education

DOI: https://doi.org/10.1109/msp.2020.2971123

IF: 15.204

2020-03-01

IEEE Signal Processing Magazine

Abstract:Prospective authors are requested to submit new, unpublished manuscripts for inclusion in the upcoming event described in this call for papers.

engineering, electrical & electronic
Data Science Principles for Interpretable and Explainable AI

Kris Sankaran

2024-08-18

Abstract:Society's capacity for algorithmic problem-solving has never been greater. Artificial Intelligence is now applied across more domains than ever, a consequence of powerful abstractions, abundant data, and accessible software. As capabilities have expanded, so have risks, with models often deployed without fully understanding their potential impacts. Interpretable and interactive machine learning aims to make complex models more transparent and controllable, enhancing user agency. This review synthesizes key principles from the growing literature in this field. We first introduce precise vocabulary for discussing interpretability, like the distinction between glass box and explainable models. We then explore connections to classical statistical and design principles, like parsimony and the gulfs of interaction. Basic explainability techniques -- including learned embeddings, integrated gradients, and concept bottlenecks -- are illustrated with a simple case study. We also review criteria for objectively evaluating interpretability approaches. Throughout, we underscore the importance of considering audience goals when designing interactive data-driven systems. Finally, we outline open challenges and discuss the potential role of data science in addressing them. Code to reproduce all examples can be found at <a class="link-external link-https" href="https://go.wisc.edu/3k1ewe" rel="external noopener nofollow">this https URL</a>.

Machine Learning
Advances in Explainable, Fair, and Trustworthy AI

Sheikh Rabiul Islam,Ingrid Russell,William Eberle,Douglas Talbert,Md Golam Moula Mehedi Hasan

DOI: https://doi.org/10.1142/s0218213024030015

IF: 1.1

2024-04-25

International Journal on Artificial Intelligence Tools

Abstract:International Journal on Artificial Intelligence Tools, Volume 33, Issue 03, May 2024. This special issue encapsulates the multifaceted landscape of contemporary challenges and innovations in Artificial Intelligence (AI) and Machine Learning (ML), with a particular focus on issues related to explainability, fairness, and trustworthiness. The exploration begins with the computational intricacies of understanding and explaining the behavior of binary neurons within neural networks. Simultaneously, ethical dimensions in AI are scrutinized, emphasizing the nuanced considerations required in defining autonomous ethical agents. The pursuit of fairness is exemplified through frameworks and methodologies in machine learning, addressing biases and promoting trust, particularly in predictive policing systems. Human-agent interaction dynamics are elucidated, revealing the nuanced relationship between task allocation, performance, and user satisfaction. The imperative of interpretability in complex predictive models is highlighted, emphasizing a query-driven methodology. Lastly, in the context of trauma triage, the study underscores the delicate trade-off between model accuracy and practitioner-friendly interpretability, introducing innovative strategies to address biases and trust-related metrics.

computer science, artificial intelligence, interdisciplinary applications
Explainability Is in the Mind of the Beholder: Establishing the Foundations of Explainable Artificial Intelligence

Kacper Sokol,Peter Flach

DOI: https://doi.org/10.48550/arXiv.2112.14466

2022-09-09

Abstract:Explainable artificial intelligence and interpretable machine learning are research domains growing in importance. Yet, the underlying concepts remain somewhat elusive and lack generally agreed definitions. While recent inspiration from social sciences has refocused the work on needs and expectations of human recipients, the field still misses a concrete conceptualisation. We take steps towards addressing this challenge by reviewing the philosophical and social foundations of human explainability, which we then translate into the technological realm. In particular, we scrutinise the notion of algorithmic black boxes and the spectrum of understanding determined by explanatory processes and explainees' background knowledge. This approach allows us to define explainability as (logical) reasoning applied to transparent insights (into, possibly black-box, predictive systems) interpreted under background knowledge and placed within a specific context -- a process that engenders understanding in a selected group of explainees. We then employ this conceptualisation to revisit strategies for evaluating explainability as well as the much disputed trade-off between transparency and predictive power, including its implications for ante-hoc and post-hoc techniques along with fairness and accountability established by explainability. We furthermore discuss components of the machine learning workflow that may be in need of interpretability, building on a range of ideas from human-centred explainability, with a particular focus on explainees, contrastive statements and explanatory processes. Our discussion reconciles and complements current research to help better navigate open questions -- rather than attempting to address any individual issue -- thus laying a solid foundation for a grounded discussion and future progress of explainable artificial intelligence and interpretable machine learning.

Artificial Intelligence,Machine Learning
Special issue on Explainable Artificial Intelligence (XAI)

Tim Miller,Robert Hoffman,Ofra Amir,Andreas Holzinger

DOI: https://doi.org/10.1016/j.artint.2022.103705

IF: 14.4

2022-06-01

Artificial Intelligence

Abstract:The special issue on Explainable Artificial Intelligence (XAI) provides a representative snapshot of the state of the art in the 2020-2021 time-frame and highlights future research directions. The scope of the special issue is intentionally broad, ranging from technical contributions to human-centered studies, from surveys to philosophical perspectives. A total of 97 papers were submitted for the special issue, of which 27 were finally accepted for publication after thorough peer review.

computer science, artificial intelligence
Science Communications for Explainable Artificial Intelligence

Simon Hudson,Matija Franklin

DOI: https://doi.org/10.48550/arXiv.2308.16377

IF: 6.4588

2023-08-31

Human-Computer Interaction

Abstract:Artificial Intelligence (AI) has a communication problem. XAI methods have been used to make AI more understandable and helped resolve some of the transparency issues that inhibit AI's broader usability. However, user evaluation studies reveal that the often numerical explanations provided by XAI methods have not always been effective for many types of users of AI systems. This article aims to adapt the major communications models from Science Communications into a framework for practitioners to understand, influence, and integrate the context of audiences both for their communications supporting AI literacy in the public and in designing XAI systems that are more adaptive to different users.
Machine Learning Interpretability: A Survey on Methods and Metrics

Diogo V. Carvalho,Eduardo M. Pereira,Jaime S. Cardoso

DOI: https://doi.org/10.3390/electronics8080832

IF: 2.9

2019-07-26

Electronics

Abstract:Machine learning systems are becoming increasingly ubiquitous. These systems’s adoption has been expanding, accelerating the shift towards a more algorithmic society, meaning that algorithmically informed decisions have greater potential for significant social impact. However, most of these accurate decision support systems remain complex black boxes, meaning their internal logic and inner workings are hidden to the user and even experts cannot fully understand the rationale behind their predictions. Moreover, new regulations and highly regulated domains have made the audit and verifiability of decisions mandatory, increasing the demand for the ability to question, understand, and trust machine learning systems, for which interpretability is indispensable. The research community has recognized this interpretability problem and focused on developing both interpretable models and explanation methods over the past few years. However, the emergence of these methods shows there is no consensus on how to assess the explanation quality. Which are the most suitable metrics to assess the quality of an explanation? The aim of this article is to provide a review of the current state of the research field on machine learning interpretability while focusing on the societal impact and on the developed methods and metrics. Furthermore, a complete literature review is presented in order to identify future directions of work on this field.

engineering, electrical & electronic,computer science, information systems,physics, applied
Editorial Special Issue on Explainable and Generalizable Deep Learning for Medical Imaging

Tianming Liu,Dajiang Zhu,Fei Wang,Islem Rekik,Xia Hu,Dinggang Shen

DOI: https://doi.org/10.1109/tnnls.2024.3395937

IF: 14.255

2024-06-04

IEEE Transactions on Neural Networks and Learning Systems

Abstract:The rapid advancements in deep learning technologies have profoundly influenced the field of medical image analysis, yet their full integration into clinical radiology practices has not progressed as quickly as expected. A significant hurdle to their widespread adoption among radiologists and clinicians is the prevailing lack of trust and confidence in the outcomes produced by these technologies. This concern primarily stems from concerns regarding the explainability and generalizability of deep learning models within the realm of medical imaging. As part of the responses from the Medical Image Analysis Community to address these critical issues, we organized the IEEE Transactions on Neural Networks and Learning Systems (TNNLS) Special Issue on explainable and generalizable deep learning for medical imaging. This IEEE TNNLS Special Issue calls for original and innovative methodological contributions that aim to address the key challenges on explainability and generalizability of deep learning for medical imaging. This IEEE TNNLS Special Issue emphasizes the research and advanced development of the technical aspects of new image analysis methodologies, and all the developed new methods should also be evaluated or validated on real and large-scale medical imaging data.

computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, hardware & architecture
Guest Editorial: Special Issue on Explainable Representation Learning-Based Intelligent Inspection and Maintenance of Complex Systems

Zhigang Liu,Cesare Alippi,Hongtian Chen,Derong Liu

DOI: https://doi.org/10.1109/tnnls.2024.3371593

IF: 14.255

2024-05-04

IEEE Transactions on Neural Networks and Learning Systems

Abstract:Over the past decade, representation learning has received particular attention in the intelligent inspection and maintenance of complex systems thanks to its overwhelming advantages in discovering and mining hidden knowledge representations. The room for in-depth investigations of representation learning-related topics remains open, especially explainable approaches for intelligent inspection and maintenance of complex systems. The primary objective of this special issue, entitled "Explainable Representation Learning-based Intelligent Inspection and Maintenance of Complex Systems," of IEEE Transactions on Neural Networks and Learning Systems is to provide the related latest achievements made by researchers and practitioners on the one hand and to identify critical issues and challenges for future investigation on the other hand.

computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, hardware & architecture
Explainable Artificial Intelligence and Machine Learning: A reality rooted perspective

Frank Emmert-Streib,Olli Yli-Harja,Matthias Dehmer

DOI: https://doi.org/10.1002/widm.1368

2020-01-26

Abstract:We are used to the availability of big data generated in nearly all fields of science as a consequence of technological progress. However, the analysis of such data possess vast challenges. One of these relates to the explainability of artificial intelligence (AI) or machine learning methods. Currently, many of such methods are non-transparent with respect to their working mechanism and for this reason are called black box models, most notably deep learning methods. However, it has been realized that this constitutes severe problems for a number of fields including the health sciences and criminal justice and arguments have been brought forward in favor of an explainable AI. In this paper, we do not assume the usual perspective presenting explainable AI as it should be, but rather we provide a discussion what explainable AI can be. The difference is that we do not present wishful thinking but reality grounded properties in relation to a scientific theory beyond physics.

Artificial Intelligence,Computer Vision and Pattern Recognition,Machine Learning
Towards A Rigorous Science of Interpretable Machine Learning

Finale Doshi-Velez,Been Kim

2017-03-03

Abstract:As machine learning systems become ubiquitous, there has been a surge of interest in interpretable machine learning: systems that provide explanation for their outputs. These explanations are often used to qualitatively assess other criteria such as safety or non-discrimination. However, despite the interest in interpretability, there is very little consensus on what interpretable machine learning is and how it should be measured. In this position paper, we first define interpretability and describe when interpretability is needed (and when it is not). Next, we suggest a taxonomy for rigorous evaluation and expose open questions towards a more rigorous science of interpretable machine learning.

Machine Learning,Artificial Intelligence
A manifesto on explainability for artificial intelligence in medicine

Carlo Combi,Beatrice Amico,Riccardo Bellazzi,Andreas Holzinger,Marinka Zitnik,Jason H. Moore,John H. Holmes

DOI: https://doi.org/10.1016/j.artmed.2022.102423

IF: 7.011

2022-11-01

Artificial Intelligence in Medicine

Abstract:The rapid increase of interest in, and use of, artificial intelligence (AI) in computer applications has raised a parallel concern about its ability (or lack thereof) to provide understandable, or explainable, output to users. This concern is especially legitimate in biomedical contexts, where patient safety is of paramount importance. This position paper brings together seven researchers working in the field with different roles and perspectives, to explore in depth the concept of explainable AI, or XAI, offering a functional definition and conceptual framework or model that can be used when considering XAI. This is followed by a series of desiderata for attaining explainability in AI, each of which touches upon a key domain in biomedicine.

engineering, biomedical,computer science, artificial intelligence,medical informatics
Interpretable and explainable machine learning: A methods‐centric overview with concrete examples

Ričards Marcinkevičs,Julia E. Vogt

DOI: https://doi.org/10.1002/widm.1493

2023-03-02

Abstract:Interpretability and explainability are essential principles of machine learning model and method design and development for medicine, economics, law, and natural sciences applications. Over the last 30 years, many techniques motivated by these properties have been developed. This review is intended for a general machine learning audience interested in exploring the challenges of interpretation and explanation beyond the logistic regression or random forest variable importance. We will examine inductive biases behind interpretable and explainable machine learning and illustrate them with concrete examples from the literature. Interpretability and explainability are crucial for machine learning (ML) and statistical applications in medicine, economics, law, and natural sciences and form an essential principle for ML model design and development. Although interpretability and explainability have escaped a precise and universal definition, many models and techniques motivated by these properties have been developed over the last 30 years, with the focus currently shifting toward deep learning. We will consider concrete examples of state‐of‐the‐art, including specially tailored rule‐based, sparse, and additive classification models, interpretable representation learning, and methods for explaining black‐box models post hoc. The discussion will emphasize the need for and relevance of interpretability and explainability, the divide between them, and the inductive biases behind the presented "zoo" of interpretable models and explanation methods. This article is categorized under: Fundamental Concepts of Data and Knowledge > Explainable AI Technologies > Machine Learning Commercial, Legal, and Ethical Issues > Social Considerations

computer science, artificial intelligence, theory & methods
Explaining Explanations: An Overview of Interpretability of Machine Learning

Leilani H. Gilpin,David Bau,Ben Z. Yuan,Ayesha Bajwa,Michael Specter,Lalana Kagal

DOI: https://doi.org/10.1109/dsaa.2018.00018

2018-10-01

Abstract:There has recently been a surge of work in explanatory artificial intelligence (XAI). This research area tackles the important problem that complex machines and algorithms often cannot provide insights into their behavior and thought processes. XAI allows users and parts of the internal system to be more transparent, providing explanations of their decisions in some level of detail. These explanations are important to ensure algorithmic fairness, identify potential bias/problems in the training data, and to ensure that the algorithms perform as expected. However, explanations produced by these systems is neither standardized nor systematically assessed. In an effort to create best practices and identify open challenges, we describe foundational concepts of explainability and show how they can be used to classify existing literature. We discuss why current approaches to explanatory methods especially for deep neural networks are insufficient. Finally, based on our survey, we conclude with suggested future research directions for explanatory artificial intelligence.
No-Convex Optimization for Signal Processing and Machine Learning: Special Issue for IEEE Signal Processing Magazine

DOI: https://doi.org/10.1109/msp.2019.2920701

IF: 15.204

2019-07-01

IEEE Signal Processing Magazine

Abstract:Prospective authors are requested to submit new, unpublished manuscripts for inclusion in the upcoming event described in this call for papers.

engineering, electrical & electronic
Flexible and Context-Specific AI Explainability: A Multidisciplinary Approach

Valérie Beaudouin,Isabelle Bloch,David Bounie,Stéphan Clémençon,Florence d'Alché-Buc,James Eagan,Winston Maxwell,Pavlo Mozharovskyi,Jayneel Parekh

DOI: https://doi.org/10.48550/arXiv.2003.07703

2020-03-13

Abstract:The recent enthusiasm for artificial intelligence (AI) is due principally to advances in deep learning. Deep learning methods are remarkably accurate, but also opaque, which limits their potential use in safety-critical applications. To achieve trust and accountability, designers and operators of machine learning algorithms must be able to explain the inner workings, the results and the causes of failures of algorithms to users, regulators, and citizens. The originality of this paper is to combine technical, legal and economic aspects of explainability to develop a framework for defining the "right" level of explain-ability in a given context. We propose three logical steps: First, define the main contextual factors, such as who the audience of the explanation is, the operational context, the level of harm that the system could cause, and the legal/regulatory framework. This step will help characterize the operational and legal needs for explanation, and the corresponding social benefits. Second, examine the technical tools available, including post hoc approaches (input perturbation, saliency maps...) and hybrid AI approaches. Third, as function of the first two steps, choose the right levels of global and local explanation outputs, taking into the account the costs involved. We identify seven kinds of costs and emphasize that explanations are socially useful only when total social benefits exceed costs.

Computers and Society
Explainability as statistical inference

Hugo Henri Joseph Senetaire,Damien Garreau,Jes Frellsen,Pierre-Alexandre Mattei

2023-12-29

Abstract:A wide variety of model explanation approaches have been proposed in recent years, all guided by very different rationales and heuristics. In this paper, we take a new route and cast interpretability as a statistical inference problem. We propose a general deep probabilistic model designed to produce interpretable predictions. The model parameters can be learned via maximum likelihood, and the method can be adapted to any predictor network architecture and any type of prediction problem. Our method is a case of amortized interpretability models, where a neural network is used as a selector to allow for fast interpretation at inference time. Several popular interpretability methods are shown to be particular cases of regularised maximum likelihood for our general model. We propose new datasets with ground truth selection which allow for the evaluation of the features importance map. Using these datasets, we show experimentally that using multiple imputation provides more reasonable interpretations.

Machine Learning,Artificial Intelligence,Methodology
Explainable AI: A Review of Machine Learning Interpretability Methods

Pantelis Linardatos,Vasilis Papastefanopoulos,Sotiris Kotsiantis

DOI: https://doi.org/10.3390/e23010018

IF: 2.738

2020-12-25

Entropy

Abstract:Recent advances in artificial intelligence (AI) have led to its widespread industrial adoption, with machine learning systems demonstrating superhuman performance in a significant number of tasks. However, this surge in performance, has often been achieved through increased model complexity, turning such systems into “black box” approaches and causing uncertainty regarding the way they operate and, ultimately, the way that they come to decisions. This ambiguity has made it problematic for machine learning systems to be adopted in sensitive yet critical domains, where their value could be immense, such as healthcare. As a result, scientific interest in the field of Explainable Artificial Intelligence (XAI), a field that is concerned with the development of new methods that explain and interpret machine learning models, has been tremendously reignited over recent years. This study focuses on machine learning interpretability methods; more specifically, a literature review and taxonomy of these methods are presented, as well as links to their programming implementations, in the hope that this survey would serve as a reference point for both theorists and practitioners.

physics, multidisciplinary
A Survey on the Explainability of Supervised Machine Learning

Nadia Burkart,Marco F. Huber

DOI: https://doi.org/10.1613/jair.1.12228

2020-11-16

Abstract:Predictions obtained by, e.g., artificial neural networks have a high accuracy but humans often perceive the models as black boxes. Insights about the decision making are mostly opaque for humans. Particularly understanding the decision making in highly sensitive areas such as healthcare or fifinance, is of paramount importance. The decision-making behind the black boxes requires it to be more transparent, accountable, and understandable for humans. This survey paper provides essential definitions, an overview of the different principles and methodologies of explainable Supervised Machine Learning (SML). We conduct a state-of-the-art survey that reviews past and recent explainable SML approaches and classifies them according to the introduced definitions. Finally, we illustrate principles by means of an explanatory case study and discuss important future directions.

Machine Learning,Artificial Intelligence
Philosophy of science at sea: Clarifying the interpretability of machine learning

Claus Beisbart,Tim Räz

DOI: https://doi.org/10.1111/phc3.12830

2022-04-19

Philosophy Compass

Abstract:In computer science, there are efforts to make machine learning more interpretable or explainable, and thus to better understand the underlying models, algorithms, and their behavior. But what exactly is interpretability, and how can it be achieved? Such questions lead into philosophical waters because their answers depend on what explanation and understanding are—and thus on issues that have been central to the philosophy of science. In this paper, we review the recent philosophical literature on interpretability. We propose a systematization in terms of four tasks for philosophers: (i) clarify the notion of interpretability, (ii) explain the value of interpretability, (iii) provide frameworks to think about interpretability, and (iv) explore important features of it to adjust our expectations about it.

IEEE Signal Processing Magazine Special Issue on Explainability in Data Science: Interpretability, Reproducibility, and Replicability

IEEE Signal Processing Magazine Special Issue: Innovation Starts with Education

Data Science Principles for Interpretable and Explainable AI

Advances in Explainable, Fair, and Trustworthy AI

Explainability Is in the Mind of the Beholder: Establishing the Foundations of Explainable Artificial Intelligence

Special issue on Explainable Artificial Intelligence (XAI)

Science Communications for Explainable Artificial Intelligence

Machine Learning Interpretability: A Survey on Methods and Metrics

Editorial Special Issue on Explainable and Generalizable Deep Learning for Medical Imaging

Guest Editorial: Special Issue on Explainable Representation Learning-Based Intelligent Inspection and Maintenance of Complex Systems

Explainable Artificial Intelligence and Machine Learning: A reality rooted perspective

Towards A Rigorous Science of Interpretable Machine Learning

A manifesto on explainability for artificial intelligence in medicine

Interpretable and explainable machine learning: A methods‐centric overview with concrete examples

Explaining Explanations: An Overview of Interpretability of Machine Learning

No-Convex Optimization for Signal Processing and Machine Learning: Special Issue for IEEE Signal Processing Magazine

Flexible and Context-Specific AI Explainability: A Multidisciplinary Approach

Explainability as statistical inference

Explainable AI: A Review of Machine Learning Interpretability Methods

A Survey on the Explainability of Supervised Machine Learning

Philosophy of science at sea: Clarifying the interpretability of machine learning