Abstract:In many AI applications today, the predominance of black-box machine learning models, due to their typically higher accuracy, amplifies the need for Explainable AI (XAI). Existing XAI approaches, such as the widely used SHAP values or counterfactual (CF) explanations, are arguably often too technical for users to understand and act upon. To enhance comprehension of explanations of AI decisions and the overall user experience, we introduce XAIstories, which leverage Large Language Models to provide narratives about how AI predictions are made: SHAPstories do so based on SHAP explanations, while CFstories do so for CF explanations. We study the impact of our approach on users' experience and understanding of AI predictions. Our results are striking: over 90% of the surveyed general audience finds the narratives generated by SHAPstories convincing. Data scientists primarily see the value of SHAPstories in communicating explanations to a general audience, with 83% of data scientists indicating they are likely to use SHAPstories for this purpose. In an image classification setting, CFstories are considered more or equally convincing as the users' own crafted stories by more than 75% of the participants. CFstories additionally bring a tenfold speed gain in creating a narrative. We also find that SHAPstories help users to more accurately summarize and understand AI decisions, in a credit scoring setting we test, correctly answering comprehension questions significantly more often than they do when only SHAP values are provided. The results thereby suggest that XAIstories may significantly help explaining and understanding AI predictions, ultimately supporting better decision-making in various applications.

What is Wrong with Language Models that Can Not Tell a Story?

Are Large Language Models Capable of Generating Human-Level Narratives?

The Next Chapter: A Study of Large Language Models in Storytelling

An Investigation about Entailment and Narrative by AI Techniques (Generative Models)

Are NLP Models Good at Tracing Thoughts: An Overview of Narrative Understanding

Evaluating Creative Short Story Generation in Humans and Large Language Models

Not (yet) the whole story: Evaluating Visual Storytelling Requires More than Measuring Coherence, Grounding, and Repetition

Assessing Language Models' Worldview for Fiction Generation

Do Language Models Enjoy Their Own Stories? Prompting Large Language Models for Automatic Story Evaluation

What Makes a Good Story and How Can We Measure It? A Comprehensive Survey of Story Evaluation

A Case Report On The "A.I. Locked-In Problem": social concerns with modern NLP

Generating Narrative Text in a Switching Dynamical System

A Precis of Language Models are not Models of Language

Tell Me a Story! Narrative-Driven XAI with Large Language Models

Towards a Formal Model of Narratives

Collaborative Storytelling with Large-scale Neural Language Models

The Language of Creativity: Evidence from Humans and Large Language Models

A Corpus for Understanding and Generating Moral Stories

Guiding Neural Story Generation with Reader Models

M-SENSE: Modeling Narrative Structure in Short Personal Narratives Using Protagonist's Mental Representations