Abstract:Deep learning models have revolutionized numerous fields, yet their decision-making processes often remain opaque, earning them the characterization of "black-box" models due to their lack of transparency and comprehensibility. This opacity presents significant challenges to understanding the rationale behind their decisions, thereby impeding their interpretability, explainability, and reliability. This review examines 718 studies published between 2015 and 2024 in high-impact journals indexed in SCI, SCI-E, SSCI, and ESCI, providing a crucial reference for researchers investigating methodologies and techniques in related domains. In this exploration, we evaluate a wide array of interpretability and explainability (XAI) strategies, including visual and feature-based explanations, local approach-based techniques, and Bayesian methods. These strategies are assessed for their effectiveness and applicability using a comprehensive set of evaluation metrics. Moving beyond traditional analyses, we propose a novel taxonomy of XAI methods, addressing gaps in the literature and offering a structured classification that elucidates the roles and interactions of these methods. Moreover, we explore the intricate relationship between interpretability and explainability, examining potential conflicts and highlighting the necessity for interpretability in practical applications. Through detailed comparative analysis, we underscore the strengths and limitations of various XAI methods across different data types, ensuring a thorough understanding of their practical performance and real-world utility. The review also examines model robustness against adversarial attacks, emphasizing the critical importance of transparency, reliability, and ethical considerations in model development. A significant emphasis is placed on identifying and mitigating biases in deep learning systems, providing insights into future research directions that aim to enhance fairness and reduce bias. By thoroughly reviewing current challenges and emerging research directions, this article equips researchers with the knowledge and tools to advance the development of more transparent, fair, and reliable deep learning systems. Ultimately, this work aims to bridge existing literature gaps by offering a forward-looking perspective that fosters innovation and progress in the field. This comprehensive review not only illuminates the current state of XAI methodologies but also contributes to the broader understanding and enhancement of deep learning systems, ensuring their ethical and equitable application across various domains.

Transferring Black-Box Decision Making to a White-Box Model

Trusting AI made decisions in healthcare by making them explainable

Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence

Unbox the black-box for the medical explainable AI via multi-modal and multi-centre data fusion: A mini-review, two showcases and beyond

Unveiling the black box: A systematic review of Explainable Artificial Intelligence in medical image analysis

Explainable Artificial Intelligence (XAI) in Biomedicine: Making AI Decisions Trustworthy for Physicians and Patients

A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI

Explainability using Decision Trees and Monte Carlo Simulations

Painting the black box white: experimental findings from applying XAI to an ECG reading setting

Unlocking the black box: an in-depth review on interpretability, explainability, and reliability in deep learning

Open the Black Box Data-Driven Explanation of Black Box Decision Systems

Unlocking the Black Box: Explainable Artificial Intelligence (XAI) for Trust and Transparency in AI Systems

Rethinking explainability: toward a postphenomenology of black-box artificial intelligence in medicine

What do we need to build explainable AI systems for the medical domain?

Explainable machine learning practices: opening another black box for reliable medical AI

Explaining Black-Box Algorithms Using Probabilistic Contrastive Counterfactuals

DiffExplainer: Unveiling Black Box Models Via Counterfactual Generation

A Brief Review of Explainable Artificial Intelligence in Healthcare

Explain To Decide: A Human-Centric Review on the Role of Explainable Artificial Intelligence in AI-assisted Decision Making

Explainable Artificial Intelligence for Medical Applications: A Review

A Grey-Box Ensemble Model Exploiting Black-Box Accuracy and White-Box Intrinsic Interpretability