Abstract:Machine Learning (ML) models are increasingly used to make critical decisions in real-world applications, yet they have become more complex, making them harder to understand. To this end, researchers have proposed several techniques to explain model predictions. However, practitioners struggle to use these explainability techniques because they often do not know which one to choose and how to interpret the results of the explanations. In this work, we address these challenges by introducing TalkToModel: an interactive dialogue system for explaining machine learning models through conversations. Specifically, TalkToModel comprises of three key components: 1) a natural language interface for engaging in conversations, making ML model explainability highly accessible, 2) a dialogue engine that adapts to any tabular model and dataset, interprets natural language, maps it to appropriate explanations, and generates text responses, and 3) an execution component that constructs the explanations. We carried out extensive quantitative and human subject evaluations of TalkToModel. Overall, we found the conversational system understands user inputs on novel datasets and models with high accuracy, demonstrating the system's capacity to generalize to new situations. In real-world evaluations with humans, 73% of healthcare workers (e.g., doctors and nurses) agreed they would use TalkToModel over baseline point-and-click systems for explainability in a disease prediction task, and 85% of ML professionals agreed TalkToModel was easier to use for computing explanations. Our findings demonstrate that TalkToModel is more effective for model explainability than existing systems, introducing a new category of explainability tools for practitioners. Code & demo released here: <a class="link-external link-https" href="https://github.com/dylan-slack/TalkToModel" rel="external noopener nofollow">this https URL</a>.

Explaining machine learning models with interactive natural language conversations using TalkToModel

TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversations

Conversations that explain predictions

Which Neural Network Makes More Explainable Decisions? an Approach Towards Measuring Explainability

VBridge: Connecting the Dots Between Features and Data to Explain Healthcare Models.

Rethinking Explainability as a Dialogue: A Practitioner's Perspective

Explaining Predictions from Machine Learning Models: Algorithms, Users, and Pedagogy

What Would You Ask the Machine Learning Model? Identification of User Needs for Model Explanations Based on Human-Model Conversations

Explainable machine learning in deployment

Model Interpretation and Explainability: Towards Creating Transparency in Prediction Models

A Model for Intelligible Interaction Between Agents That Predict and Explain

Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models

From Feature Importance to Natural Language Explanations Using LLMs with RAG

Model Explainability in Deep Learning Based Natural Language Processing

Interpretable and explainable machine learning: A methods‐centric overview with concrete examples

Stop ordering machine learning algorithms by their explainability! A user-centered investigation of performance and explainability

Teaching Meaningful Explanations

Model-Agnostic Interpretability of Machine Learning

One Explanation Does Not Fit All: The Promise of Interactive Explanations for Machine Learning Transparency

Explaining black boxes with a SMILE: Statistical Model-agnostic Interpretability with Local Explanations

Explaining Explanations in AI