Abstract:Artificial neural networks (ANNs) are powerful tools for data analysis and are particularly suitable for modeling relationships between variables for best prediction of an outcome. While these models can be used to answer many important research questions, their utility has been critically limited because the interpretation of the "black box" model is difficult. Clinical investigators usually employ ANN models to predict the clinical outcomes or to make a diagnosis; the model however is difficult to interpret for clinicians. To address this important shortcoming of neural network modeling methods, we describe several methods to help subject-matter audiences (e.g., clinicians, medical policy makers) understand neural network models. Garson's algorithm describes the relative magnitude of the importance of a descriptor (predictor) in its connection with outcome variables by dissecting the model weights. The Lek's profile method explores the relationship of the outcome variable and a predictor of interest, while holding other predictors at constant values (e.g., minimum, 20th quartile, maximum). While Lek's profile was developed specifically for neural networks, partial dependence plot is a more generic version that visualize the relationship between an outcome and one or two predictors. Finally, the local interpretable model-agnostic explanations (LIME) method can show the predictions of any classification or regression, by approximating it locally with an interpretable model. R code for the implementations of these methods is shown by using example data fitted with a standard, feed-forward neural network model. We offer codes and step-by-step description on how to use these tools to facilitate better understanding of ANN.

Feedforward neural networks as statistical models: Improving interpretability through uncertainty quantification

A statistical modelling approach to feedforward neural network model selection

A Statistical-Modelling Approach to Feedforward Neural Network Model Selection

On the Effectiveness of Interpretable Feedforward Neural Network

VOICE: Variance of Induced Contrastive Explanations to quantify Uncertainty in Neural Network Interpretability

Getting a CLUE: A Method for Explaining Uncertainty Estimates

Explainability through uncertainty: Trustworthy decision-making with neural networks

Opening the Black Box of Neural Networks: Methods for Interpreting Neural Network Models in Clinical Applications

A Survey of the Interpretability Aspect of Deep Learning Models

Towards Modeling Uncertainties of Self-explaining Neural Networks via Conformal Prediction

Explainability as statistical inference

A comparative study of conformal prediction methods for valid uncertainty quantification in machine learning

Statistical tuning of artificial neural network

Uncertainty Quantification in Deep Neural Networks through Statistical Inference on Latent Space

An Interpretable Neural Network Model Through Piecewise Linear Approximation

Uncertainty Quantification of Graph Convolution Neural Network Models of Evolving Processes

A General Framework for quantifying Aleatoric and Epistemic uncertainty in Graph Neural Networks

Understanding Uncertainty in Bayesian Deep Learning

Interpretability of deep learning models: A survey of results

Fidelity of Interpretability Methods and Perturbation Artifacts in Neural Networks

Functional Bayesian Neural Networks for Model Uncertainty Quantification