Meaningful XAI Based on User-Centric Design Methodology

Winston Maxwell,Bruno Dumas

2023-08-25

Abstract:This report first takes stock of XAI-related requirements appearing in various EU directives, regulations, guidelines, and CJEU case law. This analysis of existing requirements will permit us to have a clearer vision of the purposes, the ``why'', of XAI, which we separate into five categories: contestability, empowerment/redressing information asymmetries, control over system performance, evaluation of algorithmic decisions, and public administration transparency. The analysis of legal requirements also permits us to create four categories of recipients for explainability: data science teams; human operators of the system; persons affected by algorithmic decisions, and regulators/judges/auditors. Lastly, we identify four main operational contexts for explainability: XAI for the upstream design and testing phase; XAI for human-on-the-loop control; XAI for human-in-the-loop control; and XAI for ex-post challenges and investigations.Second, we will present user-centered design methodology, which takes the purposes, the recipients and the operational context into account in order to develop optimal XAI solutions.Third, we will suggest a methodology to permit suppliers and users of high-risk AI applications to propose local XAI solutions that are effective in the sense of being ``meaningful'', for example, useful in light of the operational, safety and fundamental rights contexts. The process used to develop these ``meaningful'' XAI solutions will be based on user-centric design principles examined in the second part.Fourth, we will suggest that the European Commission issue guidelines to provide a harmonised approach to defining ``meaningful'' explanations based on the purposes, audiences and operational contexts of AI systems. These guidelines would apply to the AI Act, but also to the other EU texts requiring explanations for algorithmic systems and results.

Human-Computer Interaction,Computers and Society

What problem does this paper attempt to address?

The paper attempts to address the issue of explainability in artificial intelligence systems (AI). Specifically, it explores how to achieve meaningful algorithmic explainability through a user-centric design methodology that combines legal and human-computer interaction (HCI) approaches. The paper emphasizes the importance of tailoring explanations to the needs and task contexts of users in different application scenarios, and the critical role these explanations play in ensuring effective human control over algorithmic systems, enhancing transparency, and protecting individual rights.

The paper points out that most current research on explainable AI (XAI) primarily focuses on the needs of data scientists, while neglecting the needs of end users. Therefore, this study aims to develop effective and meaningful XAI solutions for different user groups (such as data science teams, system operators, individuals affected by algorithmic decisions, and regulators/judges/auditors) and different operational environments (such as the early design and testing phases, human-in-the-loop control, post-hoc challenges, and investigations) through a user-centric design approach.

Additionally, the paper discusses the impact of legal requirements on XAI, including provisions on algorithmic explainability in regulations such as the EU General Data Protection Regulation (GDPR) and the proposed EU AI Act. It offers specific recommendations, such as incorporating concrete "local explainability" guidelines in the proposed EU AI Act, using a user-centric design approach to measure the "meaningfulness" of explanations, and harmonizing explainable AI requirements across different EU legislation and case law.