Abstract:Machine Learning (ML) models are increasingly used to make critical decisions in real-world applications, yet they have become more complex, making them harder to understand. To this end, researchers have proposed several techniques to explain model predictions. However, practitioners struggle to use these explainability techniques because they often do not know which one to choose and how to interpret the results of the explanations. In this work, we address these challenges by introducing TalkToModel: an interactive dialogue system for explaining machine learning models through conversations. Specifically, TalkToModel comprises of three key components: 1) a natural language interface for engaging in conversations, making ML model explainability highly accessible, 2) a dialogue engine that adapts to any tabular model and dataset, interprets natural language, maps it to appropriate explanations, and generates text responses, and 3) an execution component that constructs the explanations. We carried out extensive quantitative and human subject evaluations of TalkToModel. Overall, we found the conversational system understands user inputs on novel datasets and models with high accuracy, demonstrating the system's capacity to generalize to new situations. In real-world evaluations with humans, 73% of healthcare workers (e.g., doctors and nurses) agreed they would use TalkToModel over baseline point-and-click systems for explainability in a disease prediction task, and 85% of ML professionals agreed TalkToModel was easier to use for computing explanations. Our findings demonstrate that TalkToModel is more effective for model explainability than existing systems, introducing a new category of explainability tools for practitioners. Code & demo released here: <a class="link-external link-https" href="https://github.com/dylan-slack/TalkToModel" rel="external noopener nofollow">this https URL</a>.

Explaining Predictions from Machine Learning Models: Algorithms, Users, and Pedagogy

Explainability for Machine Learning Models: From Data Adaptability to User Perception

Explaining machine learning models with interactive natural language conversations using TalkToModel

Stop ordering machine learning algorithms by their explainability! A user-centered investigation of performance and explainability

TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversations

Explaining Models: An Empirical Study of How Explanations Impact Fairness Judgment

Conversations that explain predictions

Explainable machine learning in deployment

Model Interpretation and Explainability: Towards Creating Transparency in Prediction Models

Explaining Explanations in AI

Evaluating Explainability in Machine Learning Predictions through Explainer-Agnostic Metrics

Explainability in Machine Learning: a Pedagogical Perspective

Model-Agnostic Interpretability of Machine Learning

Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models

The Automated Laplacean Demon: How ML Challenges Our Views on Prediction and Explanation

Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations

Accountable and Explainable Methods for Complex Reasoning over Text

Teaching Meaningful Explanations

Shedding Light on Black Box Machine Learning Algorithms: Development of an Axiomatic Framework to Assess the Quality of Methods that Explain Individual Predictions

Explainability of Artificial Intelligence Models: Technical Foundations and Legal Principles

Explaining Explanations: An Overview of Interpretability of Machine Learning