Towards Explainable Goal Recognition Using Weight of Evidence (WoE): A Human-Centered Approach

Abeer Alshehri,Amal Abdulrahman,Hajar Alamri,Tim Miller,Mor Vered

2024-09-18

Abstract:Goal recognition (GR) involves inferring an agent's unobserved goal from a sequence of observations. This is a critical problem in AI with diverse applications. Traditionally, GR has been addressed using 'inference to the best explanation' or abduction, where hypotheses about the agent's goals are generated as the most plausible explanations for observed behavior. Alternatively, some approaches enhance interpretability by ensuring that an agent's behavior aligns with an observer's expectations or by making the reasoning behind decisions more transparent. In this work, we tackle a different challenge: explaining the GR process in a way that is comprehensible to humans. We introduce and evaluate an explainable model for goal recognition (GR) agents, grounded in the theoretical framework and cognitive processes underlying human behavior explanation. Drawing on insights from two human-agent studies, we propose a conceptual framework for human-centered explanations of GR. Using this framework, we develop the eXplainable Goal Recognition (XGR) model, which generates explanations for both why and why not questions. We evaluate the model computationally across eight GR benchmarks and through three user studies. The first study assesses the efficiency of generating human-like explanations within the Sokoban game domain, the second examines perceived explainability in the same domain, and the third evaluates the model's effectiveness in aiding decision-making in illegal fishing detection. Results demonstrate that the XGR model significantly enhances user understanding, trust, and decision-making compared to baseline models, underscoring its potential to improve human-agent collaboration.

Artificial Intelligence

What problem does this paper attempt to address?

### What problem does this paper attempt to solve?

This paper aims to address the issue of explainability in the Goal Recognition (GR) process. Specifically, the research team developed a model called "Explainable Goal Recognition (XGR)" to enable humans to understand the GR process. The main contributions of this paper are as follows:

1. **Proposing a Conceptual Framework**:
- The researchers proposed a conceptual framework by analyzing how humans explain GR tasks in different scenarios.
- The framework includes 11 key concepts used by people when reasoning about goal prediction and incorporates folk theory of mind and behavior to ensure the explanations align with human cognitive patterns.

2. **Developing the XGR Model**:
- Based on the aforementioned conceptual framework, the researchers constructed an XGR model that uses Weight of Evidence (WoE) to generate explanations about why and why not.
- The model can generate human-like explanations in various benchmark tests and has been validated through user studies.

3. **Experimental Validation**:
- The paper validates the effectiveness of the XGR model through three user studies:
1. Evaluating the model's efficiency in generating human-like explanations in the Sokoban game;
2. Assessing users' perceptions of the model's explanatory capabilities;
3. Verifying the model's decision support effectiveness in illegal fishing detection.

In summary, the focus of this paper is to enhance the transparency and explainability of GR systems, enabling them not only to accurately predict goals but also to clearly explain the reasons behind the predictions, thereby increasing user trust and supporting decision-making.