Black Box Model Explanations and the Human Interpretability Expectations -- An Analysis in the Context of Homicide Prediction

José Ribeiro,Níkolas Carneiro,Ronnie Alves

2024-07-04

Abstract:Strategies based on Explainable Artificial Intelligence (XAI) have promoted better human interpretability of the results of black box models. This opens up the possibility of questioning whether explanations created by XAI methods meet human expectations. The XAI methods being currently used (Ciu, Dalex, Eli5, Lofo, Shap, and Skater) provide various forms of explanations, including global rankings of relevance of features, which allow for an overview of how the model is explained as a result of its inputs and outputs. These methods provide for an increase in the explainability of the model and a greater interpretability grounded on the context of the problem. Intending to shed light on the explanations generated by XAI methods and their interpretations, this research addresses a real-world classification problem related to homicide prediction, already peer-validated, replicated its proposed black box model and used 6 different XAI methods to generate explanations and 6 different human experts. The results were generated through calculations of correlations, comparative analysis and identification of relationships between all ranks of features produced. It was found that even though it is a model that is difficult to explain, 75\% of the expectations of human experts were met, with approximately 48\% agreement between results from XAI methods and human experts. The results allow for answering questions such as: "Are the Expectation of Interpretation generated among different human experts similar?", "Do the different XAI methods generate similar explanations for the proposed problem?", "Can explanations generated by XAI methods meet human expectation of Interpretations?", and "Can Explanations and Expectations of Interpretation work together?".

Machine Learning,Artificial Intelligence

What problem does this paper attempt to address?

The main problem this paper attempts to address is evaluating whether the explanations generated by Explainable Artificial Intelligence (XAI) methods meet the expectations of human experts, and exploring the reliability and effectiveness of these explanations in practical applications. Specifically, the paper addresses this issue through the following aspects:

1. **Evaluating the consistency of XAI methods**: The study examines whether different XAI methods (such as CIU, Dalex, Eli5, Lofo, SHAP, and Skater) can generate similar explanations when addressing the same problem. This helps to understand the reliability and consistency of different XAI methods in explaining black-box models.

2. **Comparing human experts' expectations**: By inviting 6 experts from different fields to interpret the model, the study evaluates whether their expectations for explanations are consistent. This helps to understand the degree of consensus among different experts when explaining the same model.

3. **Validating the consistency between XAI methods and human experts' expectations**: By comparing the explanations generated by XAI methods with the expectations of human experts, the study evaluates whether XAI methods can meet the explanatory needs of human experts. This helps to understand the effectiveness and reliability of XAI methods in practical applications.

4. **Combining XAI methods and human experts' explanations**: The paper proposes a technique called ConeXi, which combines the explanations generated by XAI methods with the expectations of human experts to construct a comprehensive model explanation. This helps to improve the overall quality and credibility of model explanations.

The paper selects a specific real-world classification problem—homicide prediction—as the research subject. Through this problem, the paper explores the application of XAI methods in sensitive areas and how to improve the interpretability and credibility of models by combining technical means and human expertise.