Abstract:Machine Learning (ML) models are increasingly used to make critical decisions in real-world applications, yet they have become more complex, making them harder to understand. To this end, researchers have proposed several techniques to explain model predictions. However, practitioners struggle to use these explainability techniques because they often do not know which one to choose and how to interpret the results of the explanations. In this work, we address these challenges by introducing TalkToModel: an interactive dialogue system for explaining machine learning models through conversations. Specifically, TalkToModel comprises of three key components: 1) a natural language interface for engaging in conversations, making ML model explainability highly accessible, 2) a dialogue engine that adapts to any tabular model and dataset, interprets natural language, maps it to appropriate explanations, and generates text responses, and 3) an execution component that constructs the explanations. We carried out extensive quantitative and human subject evaluations of TalkToModel. Overall, we found the conversational system understands user inputs on novel datasets and models with high accuracy, demonstrating the system's capacity to generalize to new situations. In real-world evaluations with humans, 73% of healthcare workers (e.g., doctors and nurses) agreed they would use TalkToModel over baseline point-and-click systems for explainability in a disease prediction task, and 85% of ML professionals agreed TalkToModel was easier to use for computing explanations. Our findings demonstrate that TalkToModel is more effective for model explainability than existing systems, introducing a new category of explainability tools for practitioners. Code & demo released here: <a class="link-external link-https" href="https://github.com/dylan-slack/TalkToModel" rel="external noopener nofollow">this https URL</a>.

Conversations that explain predictions

TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversations

Explaining machine learning models with interactive natural language conversations using TalkToModel

Explaining Predictions from Machine Learning Models: Algorithms, Users, and Pedagogy

What Would You Ask the Machine Learning Model? Identification of User Needs for Model Explanations Based on Human-Model Conversations

Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models

Rethinking Explainability as a Dialogue: A Practitioner's Perspective

Model Interpretation and Explainability: Towards Creating Transparency in Prediction Models

Explingo: Explaining AI Predictions using Large Language Models

May I Ask a Follow-up Question? Understanding the Benefits of Conversations in Neural Network Explainability

A Model for Intelligible Interaction Between Agents That Predict and Explain

One Explanation Does Not Fit All: The Promise of Interactive Explanations for Machine Learning Transparency

From Feature Importance to Natural Language Explanations Using LLMs with RAG

Explaining Speech Classification Models via Word-Level Audio Segments and Paralinguistic Features

From Black Boxes to Conversations: Incorporating XAI in a Conversational Agent

SUBPLEX: Towards a Better Understanding of Black Box Model Explanations at the Subpopulation Level

Algorithms for interpretable machine learning

SynthTree: Co-supervised Local Model Synthesis for Explainable Prediction

Interpretable and explainable machine learning: A methods‐centric overview with concrete examples

Pyreal: A Framework for Interpretable ML Explanations

Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations