People Attribute Purpose to Autonomous Vehicles When Explaining Their Behavior

Balint Gyevnar,Stephanie Droop,Tadeg Quillien,Shay B. Cohen,Neil R. Bramley,Christopher G. Lucas,Stefano V. Albrecht

2024-05-01

Abstract:Cognitive science can help us understand which explanations people might expect, and in which format they frame these explanations, whether causal, counterfactual, or teleological (i.e., purpose-oriented). Understanding the relevance of these concepts is crucial for building good explainable AI (XAI) which offers recourse and actionability. Focusing on autonomous driving, a complex decision-making domain, we report empirical data from two surveys on (i) how people explain the behavior of autonomous vehicles in 14 unique scenarios (N1=54), and (ii) how they perceive these explanations in terms of complexity, quality, and trustworthiness (N2=356). Participants deemed teleological explanations significantly better quality than counterfactual ones, with perceived teleology being the best predictor of perceived quality and trustworthiness. Neither the perceived teleology nor the quality were affected by whether the car was an autonomous vehicle or driven by a person. This indicates that people use teleology to evaluate information about not just other people but also autonomous vehicles. Taken together, our findings highlight the importance of explanations that are framed in terms of purpose rather than just, as is standard in XAI, the causal mechanisms involved. We release the 14 scenarios and more than 1,300 elicited explanations publicly as the Human Explanations for Autonomous Driving Decisions (HEADD) dataset.

Human-Computer Interaction,Artificial Intelligence,Robotics

What problem does this paper attempt to address?

The paper attempts to address the issue of understanding the types and formats of explanations people expect when interpreting the behavior of autonomous vehicles, particularly causal, counterfactual, and teleological (i.e., goal-oriented) explanations. The authors argue that understanding the relevance of these concepts is crucial for building well-explainable artificial intelligence (XAI) systems, as it helps provide remedies and actionability.

Specifically, the paper investigates the following questions through two surveys:
1. How do people explain the behavior of autonomous vehicles in 14 unique scenarios?
2. What are people's perceptions of the quality, complexity, and credibility of these explanations?

The study found that participants rated teleological explanations as higher in quality than counterfactual explanations, and perceived teleology as the best predictor of explanation quality and credibility. Additionally, there was no significant difference in participants' explanation preferences whether the vehicle was driven by an autonomous system or a human. This suggests that people consider the goals of autonomous vehicles as well as those of other people when evaluating information.

In summary, this study highlights the importance of goal-oriented explanation patterns in interpreting autonomous decision-making, rather than solely relying on traditional causal mechanisms.