Abstract:The presence of automated decision making continuously increases in today's society. Algorithms based in machine and deep learning decide how much we pay for insurance,&#160; translate our thoughts to speech, and shape our consumption of goods (via e-marketing) and knowledge (via search engines). Machine and deep learning models are ubiquitous in science too, in particular, many promising examples are being developed to prove their feasibility for earth sciences applications, like finding temporal trends or spatial patterns in data or improving parameterization schemes for climate simulations.&#160;However, most machine and deep learning applications aim to optimise performance metrics (for instance, accuracy, which stands for the times the model prediction was right), which are rarely good indicators of trust (i.e., why these predictions were right?). In fact, with the increase of data volume and model complexity, machine learning and deep learning&#160; predictions can be very accurate but also prone to rely on spurious correlations, encode and magnify bias, and draw conclusions that do not incorporate the underlying dynamics governing the system. Because of that, the uncertainty of the predictions and our confidence in the model are difficult to estimate and the relation between inputs and outputs becomes hard to interpret.&#160;Since it is challenging to shift a community from &#8220;black&#8221; to &#8220;glass&#8221; boxes, it is more useful to implement Explainable Artificial Intelligence (XAI) techniques right at the beginning of the machine learning and deep learning adoption rather than trying to fix fundamental problems later. The good news is that most of the popular XAI techniques basically are sensitivity analyses because they consist of a systematic perturbation of some model components in order to observe how it affects the model predictions. The techniques comprise random sampling, Monte-Carlo simulations, and ensemble runs, which are common methods in geosciences. Moreover, many XAI techniques are reusable because they are model-agnostic and must be applied after the model has been fitted. In addition, interpretability provides robust arguments when communicating machine and deep learning predictions to scientists and decision-makers.In order to assist not only the practitioners but also the end-users in the evaluation of&#160; machine and deep learning results, we will explain the intuition behind some popular techniques of XAI and aleatory and epistemic Uncertainty Quantification: (1) the Permutation Importance and Gaussian processes on the inputs (i.e., the perturbation of the model inputs), (2) the Monte-Carlo Dropout, Deep ensembles, Quantile Regression, and Gaussian processes on the weights (i.e, the perturbation of the model architecture), (3) the Conformal Predictors (useful to estimate the confidence interval on the outputs), and (4) the Layerwise Relevance Propagation (LRP), Shapley values, and Local Interpretable Model-Agnostic Explanations (LIME) (designed to visualize how each feature in the data affected a particular prediction). We will also introduce some best-practises, like the detection of anomalies in the training data before the training, the implementation of fallbacks when the prediction is not reliable, and physics-guided learning by including constraints in the loss function to avoid physical inconsistencies, like the violation of conservation laws.&#160;

"Why Should You Trust My Explanation?" Understanding Uncertainty in LIME Explanations

BMB-LIME: LIME with modeling local nonlinearity and uncertainty in explainability

"How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations

Sanity Checks for Explanation Uncertainty

Cycles of Thought: Measuring LLM Confidence through Stable Explanations

Interpretability and Transparency of Machine Learning in File Fragment Analysis with Explainable Artificial Intelligence

Model-Agnostic Interpretability of Machine Learning

GLIME: General, Stable and Local LIME Explanation

An Extension of LIME with Improvement of Interpretability and Fidelity

Explainable empirical risk minimization

Making Fair ML Software using Trustworthy Explanation

Are Your Explanations Reliable? Investigating the Stability of LIME in Explaining Text Classifiers by Marrying XAI and Adversarial Attack

Comprehension Is a Double-Edged Sword: Over-Interpreting Unspecified Information in Intelligible Machine Learning Explanations

Uncertainty Quantification and Explainable Artificial Intelligence

Quantifying Uncertainty in Natural Language Explanations of Large Language Models

Explaining machine learning models using entropic variable projection

Locally Invariant Explanations: Towards Stable and Unidirectional Explanations through Local Invariant Learning

Local Interpretable Model Agnostic Shap Explanations for machine learning models

DLIME: A Deterministic Local Interpretable Model-Agnostic Explanations Approach for Computer-Aided Diagnosis Systems

Do Explanations Reflect Decisions? A Machine-centric Strategy to Quantify the Performance of Explainability Algorithms

Investigating the Impact of Model Instability on Explanations and Uncertainty