Generating Process-Centric Explanations to Enable Contestability in Algorithmic Decision-Making: Challenges and Opportunities

Mireia Yurrita,Agathe Balayn,Ujwal Gadiraju

2023-05-01

Abstract:Human-AI decision making is becoming increasingly ubiquitous, and explanations have been proposed to facilitate better Human-AI interactions. Recent research has investigated the positive impact of explanations on decision subjects' fairness perceptions in algorithmic decision-making. Despite these advances, most studies have captured the effect of explanations in isolation, considering explanations as ends in themselves, and reducing them to technical solutions provided through XAI methodologies. In this vision paper, we argue that the effect of explanations on fairness perceptions should rather be captured in relation to decision subjects' right to contest such decisions. Since contestable AI systems are open to human intervention throughout their lifecycle, contestability requires explanations that go beyond outcomes and also capture the rationales that led to the development and deployment of the algorithmic system in the first place. We refer to such explanations as process-centric explanations. In this work, we introduce the notion of process-centric explanations and describe some of the main challenges and research opportunities for generating and evaluating such explanations.

Human-Computer Interaction

What problem does this paper attempt to address?

The problem this paper attempts to address is the effectiveness and contestability of explanations in the algorithmic decision-making process. Specifically, the researchers believe that while current explanation methods help improve the transparency of algorithmic decisions, these explanations are often viewed in isolation and typically focus only on the final outcome (i.e., outcome explanations), neglecting the decision context and rationale during the development and deployment of the algorithmic system. Therefore, the paper proposes a new concept—"process-centric explanations," which not only include the results of algorithmic decisions but also cover the processes leading to these decisions and the reasons behind them. In this way, decision subjects can better understand the entire process of algorithmic decision-making and have the ability to challenge it.

The paper emphasizes that in algorithmic decision-making, explanations should not only make the decision process transparent but also grant decision subjects the right to question and refute, which is particularly important under the concept of procedural justice. Additionally, the research explores the challenges and opportunities in generating and evaluating such process-centric explanations, aiming to promote the fairness and accountability of algorithmic decision-making systems.