Abstract:Deep neural networks (DNNs) have been shown to outperform traditional machine learning algorithms in a broad variety of application domains due to their effectiveness in modeling complex problems and handling high-dimensional datasets. Many real-life datasets, however, are of increasingly high dimensionality, where a large number of features may be irrelevant for both supervised and unsupervised learning tasks. The inclusion of such features would not only introduce unwanted noise but also increase computational complexity. Furthermore, due to high non-linearity and dependency among a large number of features, DNN models tend to be unavoidably opaque and perceived as black-box methods because of their not well-understood internal functioning. Their algorithmic complexity is often simply beyond the capacities of humans to understand the interplay among myriads of hyperparameters. A well-interpretable model can identify statistically significant features and explain the way they affect the model's outcome. In this paper, we propose an efficient method to improve the interpretability of black-box models for classification tasks in the case of high-dimensional datasets. First, we train a black-box model on a high-dimensional dataset to learn the embeddings on which the classification is performed. To decompose the inner working principles of the black-box model and to identify top-k important features, we employ different probing and perturbing techniques. We then approximate the behavior of the black-box model by means of an interpretable surrogate model on the top-k feature space. Finally, we derive decision rules and local explanations from the surrogate model to explain individual decisions. Our approach outperforms state-of-the-art methods like TabNet and XGboost when tested on different datasets with varying dimensionality between 50 and 20,000 w.r.t metrics and explainability.

Opening the Black Box of Neural Networks: Methods for Interpreting Neural Network Models in Clinical Applications

Shedding Light on the Black Box: Explaining Deep Neural Network Prediction of Clinical Outcomes

Illuminating the Black Box: Interpreting Deep Neural Network Models for Psychiatric Research

A Gentle Introduction to Artificial Neural Networks

Opening the black box: interpretable machine learning for predictor finding of metabolic syndrome

Transparency of deep neural networks for medical image analysis: A review of interpretability methods

Explaining the black-box model: A survey of local interpretation methods for deep neural networks

An Interpretable Probabilistic Approach for Demystifying Black-box Predictive Models

A Survey of the Interpretability Aspect of Deep Learning Models

Interpretable Clinical Prediction Via Attention-Based Neural Network.

A Survey on Neural Network Interpretability

Interpret Neural Networks by Extracting Critical Subnetworks

Interpreting the Black Box of Supervised Learning Models: Visualizing the Impacts of Features on Prediction

Opening the Black Box: Interpretable Machine Learning for Geneticists.

Global and local interpretability techniques of supervised machine learning black box models for numerical medical data

Interpretability of deep learning models: A survey of results

Exact and Consistent Interpretation for Piecewise Linear Neural Networks: A Closed Form Solution

Interpreting Black-box Machine Learning Models for High Dimensional Datasets

An Interpretable Neural Network Model Through Piecewise Linear Approximation

Rationale production to support clinical decision-making

DiffExplainer: Unveiling Black Box Models Via Counterfactual Generation