Explainable and Human-Grounded AI for Decision Support Systems: The Theory of Epistemic Quasi-Partnerships

John Dorsch,Maximilian Moll
2024-09-23
Abstract:In the context of AI decision support systems (AI-DSS), we argue that meeting the demands of ethical and explainable AI (XAI) is about developing AI-DSS to provide human decision-makers with three types of human-grounded explanations: reasons, counterfactuals, and confidence, an approach we refer to as the RCC approach. We begin by reviewing current empirical XAI literature that investigates the relationship between various methods for generating model explanations (e.g., LIME, SHAP, Anchors), the perceived trustworthiness of the model, and end-user accuracy. We demonstrate how current theories about what constitutes good human-grounded reasons either do not adequately explain this evidence or do not offer sound ethical advice for development. Thus, we offer a novel theory of human-machine interaction: the theory of epistemic quasi-partnerships (EQP). Finally, we motivate adopting EQP and demonstrate how it explains the empirical evidence, offers sound ethical advice, and entails adopting the RCC approach.
Artificial Intelligence,Emerging Technologies,Human-Computer Interaction
What problem does this paper attempt to address?
The core problem that this paper attempts to solve is: in the artificial intelligence decision - support system (AI - DSS), how to develop ethical and explainable artificial intelligence (XAI) to provide useful and easily understandable explanations for human decision - makers. Specifically, the author proposes a new theory - Epistemic Quasi - Partnerships (EQP), and advocates the Reasons, Counterfactuals, and Confidence (RCC) approach to achieve this goal. ### Background of the Paper and Problem Statement 1. **Ethical and Explainability Requirements** - In high - risk environments (such as abuse investigations in child welfare services), the application of AI - DSS is becoming more and more widespread, so it is necessary to ensure that the decision - making processes of these systems are transparent and trustworthy. - The trust issue involves not only whether the machine is truly trustworthy but also the perceived trust of the user towards the system, that is, whether the user believes the system is reliable. 2. **Deficiencies in Current XAI Research** - Existing XAI methods (such as LIME, SHAP, Anchors, etc.) can generate model explanations, but these explanations are often difficult for ordinary users to understand and apply. - There is a lack of standardized evaluation indicators, causing XAI researchers to rely on intuition to judge what is a good explanation. - Explanations are usually designed for developers rather than ordinary users. 3. **Requirements of the EU AI Act** - The EU AI Act requires that the design and development of high - risk AI systems should ensure that their operations are sufficiently transparent so that users can understand the functions and limitations of the system. ### Solutions Proposed in the Paper 1. **Epistemic Quasi - Partnerships (EQP)** - The author proposes a new theoretical framework - Epistemic Quasi - Partnerships (EQP), which aims to explain what constitutes a good human - oriented explanation and provide ethical guidance for the development of XAI. - EQP emphasizes that AI systems should be regarded as "quasi - partners" and enhance user understanding and trust by providing Reasons, Counterfactuals, and Confidence (RCC). 2. **RCC Approach** - **Reasons**: Explain the reasons why the model makes a specific prediction. - **Counterfactuals**: Show how the model prediction would change if the input conditions were changed. - **Confidence**: Clearly state the confidence level of the model in the prediction result. ### Main Contributions of the Paper - **Theoretical Innovation**: Proposes the EQP theory, filling the gaps in existing XAI research. - **Empirical Support**: Proves the effectiveness of the RCC method by reviewing empirical studies in existing literature. - **Ethical Guidance**: Provides theoretical basis and practical suggestions for developing ethical AI - DSS. In summary, this paper aims to solve the problem of difficult - to - understand explanations in existing XAI research by introducing the EQP theory and RCC method, thereby improving the transparency and credibility of AI - DSS and ultimately promoting the effectiveness of human - machine collaboration.