Exploring Evaluation Methodologies for Explainable AI: Guidelines for Objective and Subjective Assessment

Sule Tekkesinoglu
DOI: https://doi.org/10.2139/ssrn.4667052
2024-01-01
SSRN Electronic Journal
Abstract:This article explores the landscape of evaluation methodologies for Explainable Artificial Intelligence (XAI), focusing on both objective and subjective assessment paradigms. Objective evaluation methods provide structured and measurable assessments, offering insights into the system's performance, technical robustness, and functional effectiveness through quantitative metrics. These metrics are practical in benchmarking, verifying, and estimating functional effectiveness against predefined standards and existing explanation methods. Subjective assessments explore user-centric dimensions, elucidating perceptions, preferences, and interactions with AI-generated explanations. These assessment techniques unravel the human factors, capturing the user's experience of quality, satisfaction, trust, and performance. This work presents a list of evaluation metrics concerning objective and subjective properties of explanations. Additionally, concerning subjective measurements, an example of a user satisfaction questionnaire along with open-ended questions is provided. These can be tailored based on the context of the explanations and the user group being surveyed. Taken together, the outlined assessment methods provide a framework that caters to the diverse facets of XAI explainability to facilitate a deeper understanding of XAI systems' capabilities and enhance trust in AI-driven technologies.
What problem does this paper attempt to address?