Proceedings of NIPS 2017 Symposium on Interpretable Machine Learning

Andrew Gordon Wilson,Jason Yosinski,Patrice Simard,Rich Caruana,William Herlands

DOI: https://doi.org/10.48550/arXiv.1711.09889

IF: 5.414

2017-11-27

Machine Learning

Abstract:This is the Proceedings of NIPS 2017 Symposium on Interpretable Machine Learning, held in Long Beach, California, USA on December 7, 2017

What problem does this paper attempt to address?

Proceedings of the 2017 ICML Workshop on Human Interpretability in Machine Learning (WHI 2017)

Been Kim,Dmitry M. Malioutov,Kush R. Varshney,Adrian Weller

DOI: https://doi.org/10.48550/arXiv.1708.02666

2017-08-09

Abstract:This is the Proceedings of the 2017 ICML Workshop on Human Interpretability in Machine Learning (WHI 2017), which was held in Sydney, Australia, August 10, 2017. Invited speakers were Tony Jebara, Pang Wei Koh, and David Sontag.

Machine Learning
Proceedings of the 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016)

Been Kim,Dmitry M. Malioutov,Kush R. Varshney

DOI: https://doi.org/10.48550/arXiv.1607.02531

2016-07-28

Abstract:This is the Proceedings of the 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016), which was held in New York, NY, June 23, 2016. Invited speakers were Susan Athey, Rich Caruana, Jacob Feldman, Percy Liang, and Hanna Wallach.

Machine Learning
Interpretable machine learning: Fundamental principles and 10 grand challenges

Cynthia Rudin,Chaofan Chen,Zhi Chen,Haiyang Huang,Lesia Semenova,Chudi Zhong

DOI: https://doi.org/10.1214/21-ss133

2022-01-01

Statistics Surveys

Abstract:Interpretability in machine learning (ML) is crucial for high stakes decisions and troubleshooting. In this work, we provide fundamental principles for interpretable ML, and dispel common misunderstandings that dilute the importance of this crucial topic. We also identify 10 technical challenge areas in interpretable machine learning and provide history and background on each problem. Some of these problems are classically important, and some are recent problems that have arisen in the last few years. These problems are: (1) Optimizing sparse logical models such as decision trees; (2) Optimization of scoring systems; (3) Placing constraints into generalized additive models to encourage sparsity and better interpretability; (4) Modern case-based reasoning, including neural networks and matching for causal inference; (5) Complete supervised disentanglement of neural networks; (6) Complete or even partial unsupervised disentanglement of neural networks; (7) Dimensionality reduction for data visualization; (8) Machine learning models that can incorporate physics and other generative or causal constraints; (9) Characterization of the “Rashomon set” of good models; and (10) Interpretable reinforcement learning. This survey is suitable as a starting point for statisticians and computer scientists interested in working in interpretable machine learning.

English Else
Proceedings of the 5th Workshop on Machine Learning and Interpretation in Neuroimaging (MLINI) at NIPS 2015

I. Rish,L. Wehbe,G. Langs,M. Grosse-Wentrup,B. Murphy,G. Cecchi

DOI: https://doi.org/10.48550/arXiv.1605.04435

2016-05-15

Abstract:This volume is a collection of contributions from the 5th Workshop on Machine Learning and Interpretation in Neuroimaging (MLINI) at the Neural Information Processing Systems (NIPS 2015) conference. Modern multivariate statistical methods developed in the rapidly growing field of machine learning are being increasingly applied to various problems in neuroimaging, from cognitive state detection to clinical diagnosis and prognosis. Multivariate pattern analysis methods are designed to examine complex relationships between high-dimensional signals, such as brain images, and outcomes of interest, such as the category of a stimulus, a type of a mental state of a subject, or a specific mental disorder. Such techniques are in contrast with the traditional mass-univariate approaches that dominated neuroimaging in the past and treated each individual imaging measurement in isolation. We believe that machine learning has a prominent role in shaping how questions in neuroscience are framed, and that the machine-learning mind set is now entering modern psychology and behavioral studies. It is also equally important that practical applications in these fields motivate a rapidly evolving line or research in the machine learning community. In parallel, there is an intense interest in learning more about brain function in the context of rich naturalistic environments and scenes. Efforts to go beyond highly specific paradigms that pinpoint a single function, towards schemes for measuring the interaction with natural and more varied scene are made. The goal of the workshop is to pinpoint the most pressing issues and common challenges across the neuroscience, neuroimaging, psychology and machine learning fields, and to sketch future directions and open questions in the light of novel methodology.

Machine Learning
Proceedings of the 2018 ICML Workshop on Human Interpretability in Machine Learning (WHI 2018)

Been Kim,Kush R. Varshney,Adrian Weller

DOI: https://doi.org/10.48550/arXiv.1807.01308

2018-07-04

Abstract:This is the Proceedings of the 2018 ICML Workshop on Human Interpretability in Machine Learning (WHI 2018), which was held in Stockholm, Sweden, July 14, 2018. Invited speakers were Barbara Engelhardt, Cynthia Rudin, Fernanda Viégas, and Martin Wattenberg.

Machine Learning
Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges

Christoph Molnar,Giuseppe Casalicchio,Bernd Bischl

DOI: https://doi.org/10.1007/978-3-030-65965-3_28

2020-10-19

Abstract:We present a brief history of the field of interpretable machine learning (IML), give an overview of state-of-the-art interpretation methods, and discuss challenges. Research in IML has boomed in recent years. As young as the field is, it has over 200 years old roots in regression modeling and rule-based machine learning, starting in the 1960s. Recently, many new IML methods have been proposed, many of them model-agnostic, but also interpretation techniques specific to deep learning and tree-based ensembles. IML methods either directly analyze model components, study sensitivity to input perturbations, or analyze local or global surrogate approximations of the ML model. The field approaches a state of readiness and stability, with many methods not only proposed in research, but also implemented in open-source software. But many important challenges remain for IML, such as dealing with dependent features, causal interpretation, and uncertainty estimation, which need to be resolved for its successful application to scientific problems. A further challenge is a missing rigorous definition of interpretability, which is accepted by the community. To address the challenges and advance the field, we urge to recall our roots of interpretable, data-driven modeling in statistics and (rule-based) ML, but also to consider other areas such as sensitivity analysis, causal inference, and the social sciences.

Machine Learning
What is Interpretable? Using Machine Learning to Design Interpretable Decision-Support Systems

Owen Lahav,Nicholas Mastronarde,Mihaela van der Schaar

DOI: https://doi.org/10.48550/arXiv.1811.10799

2019-06-12

Abstract:Recent efforts in Machine Learning (ML) interpretability have focused on creating methods for explaining black-box ML models. However, these methods rely on the assumption that simple approximations, such as linear models or decision-trees, are inherently human-interpretable, which has not been empirically tested. Additionally, past efforts have focused exclusively on comprehension, neglecting to explore the trust component necessary to convince non-technical experts, such as clinicians, to utilize ML models in practice. In this paper, we posit that reinforcement learning (RL) can be used to learn what is interpretable to different users and, consequently, build their trust in ML models. To validate this idea, we first train a neural network to provide risk assessments for heart failure patients. We then design a RL-based clinical decision-support system (DSS) around the neural network model, which can learn from its interactions with users. We conduct an experiment involving a diverse set of clinicians from multiple institutions in three different countries. Our results demonstrate that ML experts cannot accurately predict which system outputs will maximize clinicians' confidence in the underlying neural network model, and suggest additional findings that have broad implications to the future of research into ML interpretability and the use of ML in medicine.

Machine Learning
Proceedings of NIPS 2017 Workshop on Machine Learning for the Developing World

Maria De-Arteaga,William Herlands

DOI: https://doi.org/10.48550/arXiv.1711.09522

IF: 5.414

2017-11-27

Machine Learning

Abstract:This is the Proceedings of NIPS 2017 Workshop on Machine Learning for the Developing World, held in Long Beach, California, USA on December 8, 2017
Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond

Xuhong Li,Haoyi Xiong,Xingjian Li,Xuanyu Wu,Xiao Zhang,Ji Liu,Jiang Bian,Dejing Dou

DOI: https://doi.org/10.1007/s10115-022-01756-8

IF: 2.7

2022-09-15

Knowledge and Information Systems

Abstract:Deep neural networks have been well-known for their superb handling of various machine learning and artificial intelligence tasks. However, due to their over-parameterized black-box nature, it is often difficult to understand the prediction results of deep models. In recent years, many interpretation tools have been proposed to explain or reveal how deep models make decisions. In this paper, we review this line of research and try to make a comprehensive survey. Specifically, we first introduce and clarify two basic concepts—interpretations and interpretability—that people usually get confused about. To address the research efforts in interpretations, we elaborate the designs of a number of interpretation algorithms, from different perspectives, by proposing a new taxonomy. Then, to understand the interpretation results, we also survey the performance metrics for evaluating interpretation algorithms. Further, we summarize the current works in evaluating models' interpretability using "trustworthy" interpretation algorithms. Finally, we review and discuss the connections between deep models' interpretations and other factors, such as adversarial robustness and learning from interpretations, and we introduce several open-source libraries for interpretation algorithms and evaluation approaches.

computer science, information systems, artificial intelligence
Proceedings of the 29th International Conference on Machine Learning (ICML-12)

John Langford,Joelle Pineau

DOI: https://doi.org/10.48550/arXiv.1207.4676

IF: 5.414

2012-07-19

Machine Learning

Abstract:This is an index to the papers that appear in the Proceedings of the 29th International Conference on Machine Learning (ICML-12). The conference was held in Edinburgh, Scotland, June 27th - July 3rd, 2012.
Interpretable Symbolic Regression for Data Science: Analysis of the 2022 Competition

F. O. de Franca,M. Virgolin,M. Kommenda,M. S. Majumder,M. Cranmer,G. Espada,L. Ingelse,A. Fonseca,M. Landajuela,B. Petersen,R. Glatt,N. Mundhenk,C. S. Lee,J. D. Hochhalter,D. L. Randall,P. Kamienny,H. Zhang,G. Dick,A. Simon,B. Burlacu,Jaan Kasak,Meera Machado,Casper Wilstrup,W. G. La Cava

2023-07-03

Abstract:Symbolic regression searches for analytic expressions that accurately describe studied phenomena. The main attraction of this approach is that it returns an interpretable model that can be insightful to users. Historically, the majority of algorithms for symbolic regression have been based on evolutionary algorithms. However, there has been a recent surge of new proposals that instead utilize approaches such as enumeration algorithms, mixed linear integer programming, neural networks, and Bayesian optimization. In order to assess how well these new approaches behave on a set of common challenges often faced in real-world data, we hosted a competition at the 2022 Genetic and Evolutionary Computation Conference consisting of different synthetic and real-world datasets which were blind to entrants. For the real-world track, we assessed interpretability in a realistic way by using a domain expert to judge the trustworthiness of candidate models.We present an in-depth analysis of the results obtained in this competition, discuss current challenges of symbolic regression algorithms and highlight possible improvements for future competitions.

Machine Learning,Artificial Intelligence
Interpretable Narrative Explanation for ML Predictors with LP: A Case Study for XAI

Andrea Omicini,Roberta Calegari,Jason Dellaluce,Giovanni Ciatto

Abstract:In the era of digital revolution, individual lives are going to cross and interconnect ubiquitous online domains and offline reality based on smart technologies—discovering, storing, processing, learning, analysing, and predicting from huge amounts of environment-collected data. Sub-symbolic techniques, such as deep learning, play a key role there, yet they are often built as black boxes, which are not inspectable, interpretable, explainable. New research efforts towards explainable artificial intelligence (XAI) are trying to address those issues, with the final purpose of building understandable, accountable, and trustable AI systems—still, seemingly with a long way to go. Generally speaking, while we fully understand and appreciate the power of sub-symbolic approaches, we believe that symbolic approaches to machine intelligence, once properly combined with sub-symbolic ones, have a critical role to play in order to achieve key properties of XAI such as observability, interpretability, explainability, accountability, and trustability. In this paper we describe an example of integration of symbolic and sub-symbolic techniques. First, we sketch a general framework where symbolic and sub-symbolic approaches could fruitfully combine to produce intelligent behaviour in AI applications. Then, we focus in particular on the goal of building a narrative explanation for ML predictors: to this end, we exploit the logical knowledge obtained translating decision tree predictors into logical programs.

Psychology,Computer Science
NSL: Hybrid Interpretable Learning From Noisy Raw Data

Daniel Cunnington,Alessandra Russo,Mark Law,Jorge Lobo,Lance Kaplan

DOI: https://doi.org/10.48550/arXiv.2012.05023

2021-06-25

Abstract:Inductive Logic Programming (ILP) systems learn generalised, interpretable rules in a data-efficient manner utilising existing background knowledge. However, current ILP systems require training examples to be specified in a structured logical format. Neural networks learn from unstructured data, although their learned models may be difficult to interpret and are vulnerable to data perturbations at run-time. This paper introduces a hybrid neural-symbolic learning framework, called NSL, that learns interpretable rules from labelled unstructured data. NSL combines pre-trained neural networks for feature extraction with FastLAS, a state-of-the-art ILP system for rule learning under the answer set semantics. Features extracted by the neural components define the structured context of labelled examples and the confidence of the neural predictions determines the level of noise of the examples. Using the scoring function of FastLAS, NSL searches for short, interpretable rules that generalise over such noisy examples. We evaluate our framework on propositional and first-order classification tasks using the MNIST dataset as raw data. Specifically, we demonstrate that NSL is able to learn robust rules from perturbed MNIST data and achieve comparable or superior accuracy when compared to neural network and random forest baselines whilst being more general and interpretable.

Machine Learning,Artificial Intelligence
A Survey of the Interpretability Aspect of Deep Learning Models

Eliot Spitzer,Rona Miles

DOI: https://doi.org/10.53759/0088/jbsha202303006

2023-01-05

Journal of Biomedical and Sustainable Healthcare Applications

Abstract:Deep neural networks have attained near-human degree of quality in images, textual, audio, and video recording categorization and predictions tasks. The networks, on the other hand, are still typically thought of as black-box functional probabilistic models that transfer an input data to a trained classifier. Integrating these systems into mission-critical activities like clinical diagnosis, scheduling, and management is the next stage in this human-machine evolutionary change, and it necessitates a degree of confidence in the technology output. Statistical measures are often employed to estimate an output's volatility. The idea of trust, on the other hand, is dependent on a human's sight into a machine's inner workings. To put it another way, the neural networks must justify its outputs in a way that is intelligible to humans, leading to new insights into its internal workings. "Interpretable deep networks" is the name we give to such networks. The concept of interpretability is not one-dimensional. Indeed, the variability of an interpretation owing to varying degrees of human comprehension necessitates the existence of a plethora of characteristics that together define interpretability. Furthermore, the model's interpretations may be expressed in terms of low-level network variables or input properties. We describe several of the variables that are helpful for model interpretability in this study, as well as previous work on those dimensions. We do a gap analysis to determine what remains to be improved to increase models interpretability as step of the procedure.
Interpretability of Machine Learning Methods Applied to Neuroimaging

Elina Thibeau-Sutre,Sasha Collin,Ninon Burgos,Olivier Colliot

DOI: https://doi.org/10.48550/arXiv.2204.07005

2022-04-14

Computer Vision and Pattern Recognition

Abstract:Deep learning methods have become very popular for the processing of natural images, and were then successfully adapted to the neuroimaging field. As these methods are non-transparent, interpretability methods are needed to validate them and ensure their reliability. Indeed, it has been shown that deep learning models may obtain high performance even when using irrelevant features, by exploiting biases in the training set. Such undesirable situations can potentially be detected by using interpretability methods. Recently, many methods have been proposed to interpret neural networks. However, this domain is not mature yet. Machine learning users face two major issues when aiming to interpret their models: which method to choose, and how to assess its reliability? Here, we aim at providing answers to these questions by presenting the most common interpretability methods and metrics developed to assess their reliability, as well as their applications and benchmarks in the neuroimaging context. Note that this is not an exhaustive survey: we aimed to focus on the studies which we found to be the most representative and relevant.
Algorithms for interpretable machine learning

Cynthia Rudin

DOI: https://doi.org/10.1145/2623330.2630823

2014-08-24

Abstract:It is extremely important in many application domains to have transparency in predictive modeling. Domain experts do not tend to prefer "black box" predictive model models. They would like to understand how predictions are made, and possibly, prefer models that emulate the way a human expert might make a decision, with a few important variables, and a clear convincing reason to make a particular prediction. I will discuss recent work on interpretable predictive modeling with decision lists and sparse integer linear models. I will describe several approaches, including an algorithm based on discrete optimization, and an algorithm based on Bayesian analysis. I will show examples of interpretable models for stroke prediction in medical patients and prediction of violent crime in young people raised in out-of-home care. Collaborators are Ben Letham, Berk Ustun, Stefano Traca, Siong Thye Goh, Tyler McCormick, and David Madigan.
Proceedings of AAAI 2019 Workshop on Network Interpretability for Deep Learning

Quanshi Zhang,Lixin Fan,Bolei Zhou

DOI: https://doi.org/10.48550/arXiv.1901.08813

2020-07-29

Abstract:This is the Proceedings of AAAI 2019 Workshop on Network Interpretability for Deep Learning

Artificial Intelligence
Explainable AI: A Review of Machine Learning Interpretability Methods

Pantelis Linardatos,Vasilis Papastefanopoulos,Sotiris Kotsiantis

DOI: https://doi.org/10.3390/e23010018

IF: 2.738

2020-12-25

Entropy

Abstract:Recent advances in artificial intelligence (AI) have led to its widespread industrial adoption, with machine learning systems demonstrating superhuman performance in a significant number of tasks. However, this surge in performance, has often been achieved through increased model complexity, turning such systems into “black box” approaches and causing uncertainty regarding the way they operate and, ultimately, the way that they come to decisions. This ambiguity has made it problematic for machine learning systems to be adopted in sensitive yet critical domains, where their value could be immense, such as healthcare. As a result, scientific interest in the field of Explainable Artificial Intelligence (XAI), a field that is concerned with the development of new methods that explain and interpret machine learning models, has been tremendously reignited over recent years. This study focuses on machine learning interpretability methods; more specifically, a literature review and taxonomy of these methods are presented, as well as links to their programming implementations, in the hope that this survey would serve as a reference point for both theorists and practitioners.

physics, multidisciplinary
Towards A Rigorous Science of Interpretable Machine Learning

Finale Doshi-Velez,Been Kim

2017-03-03

Abstract:As machine learning systems become ubiquitous, there has been a surge of interest in interpretable machine learning: systems that provide explanation for their outputs. These explanations are often used to qualitatively assess other criteria such as safety or non-discrimination. However, despite the interest in interpretability, there is very little consensus on what interpretable machine learning is and how it should be measured. In this position paper, we first define interpretability and describe when interpretability is needed (and when it is not). Next, we suggest a taxonomy for rigorous evaluation and expose open questions towards a more rigorous science of interpretable machine learning.

Machine Learning,Artificial Intelligence
Meaningful Models: Utilizing Conceptual Structure to Improve Machine Learning Interpretability

Nick Condry

DOI: https://doi.org/10.48550/arXiv.1607.00279

2016-07-01

Abstract:The last decade has seen huge progress in the development of advanced machine learning models; however, those models are powerless unless human users can interpret them. Here we show how the mind's construction of concepts and meaning can be used to create more interpretable machine learning models. By proposing a novel method of classifying concepts, in terms of 'form' and 'function', we elucidate the nature of meaning and offer proposals to improve model understandability. As machine learning begins to permeate daily life, interpretable models may serve as a bridge between domain-expert authors and non-expert users.

Machine Learning,Artificial Intelligence

Proceedings of NIPS 2017 Symposium on Interpretable Machine Learning

Proceedings of the 2017 ICML Workshop on Human Interpretability in Machine Learning (WHI 2017)

Proceedings of the 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016)

Interpretable machine learning: Fundamental principles and 10 grand challenges

Proceedings of the 5th Workshop on Machine Learning and Interpretation in Neuroimaging (MLINI) at NIPS 2015

Proceedings of the 2018 ICML Workshop on Human Interpretability in Machine Learning (WHI 2018)

Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges

What is Interpretable? Using Machine Learning to Design Interpretable Decision-Support Systems

Proceedings of NIPS 2017 Workshop on Machine Learning for the Developing World

Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond

Proceedings of the 29th International Conference on Machine Learning (ICML-12)

Interpretable Symbolic Regression for Data Science: Analysis of the 2022 Competition

Interpretable Narrative Explanation for ML Predictors with LP: A Case Study for XAI

NSL: Hybrid Interpretable Learning From Noisy Raw Data

A Survey of the Interpretability Aspect of Deep Learning Models

Interpretability of Machine Learning Methods Applied to Neuroimaging

Algorithms for interpretable machine learning

Proceedings of AAAI 2019 Workshop on Network Interpretability for Deep Learning

Explainable AI: A Review of Machine Learning Interpretability Methods

Towards A Rigorous Science of Interpretable Machine Learning

Meaningful Models: Utilizing Conceptual Structure to Improve Machine Learning Interpretability