Abstract:Explainability has been a challenge in AI for as long as AI has existed. With the recently increased use of AI in society, it has become more important than ever that AI systems would be able to explain the reasoning behind their results also to end-users in situations such as being eliminated from a recruitment process or having a bank loan application refused by an AI system. Especially if the AI system has been trained using Machine Learning, it tends to contain too many parameters for them to be analysed and understood, which has caused them to be called `black-box' systems. Most Explainable AI (XAI) methods are based on extracting an interpretable model that can be used for producing explanations. However, the interpretable model does not necessarily map accurately to the original black-box model. Furthermore, the understandability of interpretable models for an end-user remains questionable. The notions of Contextual Importance and Utility (CIU) presented in this paper make it possible to produce human-like explanations of black-box outcomes directly, without creating an interpretable model. Therefore, CIU explanations map accurately to the black-box model itself. CIU is completely model-agnostic and can be used with any black-box system. In addition to feature importance, the utility concept that is well-known in Decision Theory provides a new dimension to explanations compared to most existing XAI methods. Finally, CIU can produce explanations at any level of abstraction and using different vocabularies and other means of interaction, which makes it possible to adjust explanations and interaction according to the context and to the target users.

Do intermediate feature coalitions aid explainability of black-box models?

Understanding Inter-Concept Relationships in Concept-Based Models

Model Agnostic Multilevel Explanations

Coalitional Strategies for Efficient Individual Prediction Explanation

Explanations of Black-Box Models based on Directional Feature Interactions

Approximation of group explainers with coalition structure using Monte Carlo sampling on the product space of coalitions and features

Constraint-Driven Explanations for Black-Box ML Models

Towards a Unified Framework for Evaluating Explanations

Explaining Decisions in ML Models: a Parameterized Complexity Analysis

Explainable AI without Interpretable Model

Does Dataset Complexity Matters for Model Explainers?

Succint Interaction-Aware Explanations

From Intrinsic to Counterfactual: On the Explainability of Contextualized Recommender Systems

DiConStruct: Causal Concept-based Explanations through Black-Box Distillation

CohEx: A Generalized Framework for Cohort Explanation

Crowdsourcing and Evaluating Concept-driven Explanations of Machine Learning Models

Explainable data-driven modeling via mixture of experts: towards effective blending of grey and black-box models

Explaining Black-box Model Predictions via Two-level Nested Feature Attributions with Consistency Property

Critical Empirical Study on Black-box Explanations in AI

Model Interpretation and Explainability: Towards Creating Transparency in Prediction Models

Even-if Explanations: Formal Foundations, Priorities and Complexity