Abstract:Explainable artificial intelligence and interpretable machine learning are research domains growing in importance. Yet, the underlying concepts remain somewhat elusive and lack generally agreed definitions. While recent inspiration from social sciences has refocused the work on needs and expectations of human recipients, the field still misses a concrete conceptualisation. We take steps towards addressing this challenge by reviewing the philosophical and social foundations of human explainability, which we then translate into the technological realm. In particular, we scrutinise the notion of algorithmic black boxes and the spectrum of understanding determined by explanatory processes and explainees' background knowledge. This approach allows us to define explainability as (logical) reasoning applied to transparent insights (into, possibly black-box, predictive systems) interpreted under background knowledge and placed within a specific context -- a process that engenders understanding in a selected group of explainees. We then employ this conceptualisation to revisit strategies for evaluating explainability as well as the much disputed trade-off between transparency and predictive power, including its implications for ante-hoc and post-hoc techniques along with fairness and accountability established by explainability. We furthermore discuss components of the machine learning workflow that may be in need of interpretability, building on a range of ideas from human-centred explainability, with a particular focus on explainees, contrastive statements and explanatory processes. Our discussion reconciles and complements current research to help better navigate open questions -- rather than attempting to address any individual issue -- thus laying a solid foundation for a grounded discussion and future progress of explainable artificial intelligence and interpretable machine learning.

Explainable AI: Beware of Inmates Running the Asylum Or: How I Learnt to Stop Worrying and Love the Social and Behavioural Sciences

Explanation in artificial intelligence: Insights from the social sciences

Expl(AI)ned: The Impact of Explainable Artificial Intelligence on Users' Information Processing

Psychology and AI at a Crossroads: How Might Complex Systems Explain Themselves?

Explainable AI is Dead, Long Live Explainable AI! Hypothesis-driven decision support

Explainability Is in the Mind of the Beholder: Establishing the Foundations of Explainable Artificial Intelligence

Don't be Fooled: The Misinformation Effect of Explanations in Human-AI Collaboration

Explaining Explaining

Helpful, Misleading or Confusing: How Humans Perceive Fundamental Building Blocks of Artificial Intelligence Explanations

Designing Theory-Driven User-Centric Explainable AI

The Drawback of Insight: Detailed Explanations Can Reduce Agreement with XAI

AI, Explainability and Public Reason: The Argument from the Limitations of the Human Mind

Explaining Explanations in AI

Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI

Explainable Artificial Intelligence: Concepts, Applications, Research Challenges and Visions

Explainability Pitfalls: Beyond Dark Patterns in Explainable AI

Explainable Artificial Intelligence: Evaluating the Objective and Subjective Impacts of xAI on Human-Agent Interaction

Explainable AI for Software Engineering

The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies

How to explain AI systems to end users: a systematic literature review and research agenda