Abstract:The presence of automated decision making continuously increases in today's society. Algorithms based in machine and deep learning decide how much we pay for insurance,&#160; translate our thoughts to speech, and shape our consumption of goods (via e-marketing) and knowledge (via search engines). Machine and deep learning models are ubiquitous in science too, in particular, many promising examples are being developed to prove their feasibility for earth sciences applications, like finding temporal trends or spatial patterns in data or improving parameterization schemes for climate simulations.&#160;However, most machine and deep learning applications aim to optimise performance metrics (for instance, accuracy, which stands for the times the model prediction was right), which are rarely good indicators of trust (i.e., why these predictions were right?). In fact, with the increase of data volume and model complexity, machine learning and deep learning&#160; predictions can be very accurate but also prone to rely on spurious correlations, encode and magnify bias, and draw conclusions that do not incorporate the underlying dynamics governing the system. Because of that, the uncertainty of the predictions and our confidence in the model are difficult to estimate and the relation between inputs and outputs becomes hard to interpret.&#160;Since it is challenging to shift a community from &#8220;black&#8221; to &#8220;glass&#8221; boxes, it is more useful to implement Explainable Artificial Intelligence (XAI) techniques right at the beginning of the machine learning and deep learning adoption rather than trying to fix fundamental problems later. The good news is that most of the popular XAI techniques basically are sensitivity analyses because they consist of a systematic perturbation of some model components in order to observe how it affects the model predictions. The techniques comprise random sampling, Monte-Carlo simulations, and ensemble runs, which are common methods in geosciences. Moreover, many XAI techniques are reusable because they are model-agnostic and must be applied after the model has been fitted. In addition, interpretability provides robust arguments when communicating machine and deep learning predictions to scientists and decision-makers.In order to assist not only the practitioners but also the end-users in the evaluation of&#160; machine and deep learning results, we will explain the intuition behind some popular techniques of XAI and aleatory and epistemic Uncertainty Quantification: (1) the Permutation Importance and Gaussian processes on the inputs (i.e., the perturbation of the model inputs), (2) the Monte-Carlo Dropout, Deep ensembles, Quantile Regression, and Gaussian processes on the weights (i.e, the perturbation of the model architecture), (3) the Conformal Predictors (useful to estimate the confidence interval on the outputs), and (4) the Layerwise Relevance Propagation (LRP), Shapley values, and Local Interpretable Model-Agnostic Explanations (LIME) (designed to visualize how each feature in the data affected a particular prediction). We will also introduce some best-practises, like the detection of anomalies in the training data before the training, the implementation of fallbacks when the prediction is not reliable, and physics-guided learning by including constraints in the loss function to avoid physical inconsistencies, like the violation of conservation laws.&#160;

Explainability meets uncertainty quantification: Insights from feature-based model fusion on multimodal time series

Uncertainty-Based Rejection in Machine Learning: Implications for Model Development and Interpretability

Towards a More Reliable Interpretation of Machine Learning Outputs for Safety-Critical Systems using Feature Importance Fusion

Multimodal Explainability via Latent Shift applied to COVID-19 stratification

Improving Explainability of Disentangled Representations using Multipath-Attribution Mappings

Dual Low-Rank Multimodal Fusion

Feature Importance and Explainability in Quantum Machine Learning

Feature importance to explain multimodal prediction models. A clinical use case

Uncertainty Quantification and Explainable Artificial Intelligence

Uncertainty Quantification for Gradient-based Explanations in Neural Networks

Evaluating Explainable AI on a Multi-Modal Medical Imaging Task: Can Existing Algorithms Fulfill Clinical Requirements?

Unified Explanations in Machine Learning Models: A Perturbation Approach

DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanations

Exploiting "Quantum-like Interference" in Decision Fusion for Ranking Multimodal Documents

Calibrated Explanations: with Uncertainty Information and Counterfactuals

Explainability through uncertainty: Trustworthy decision-making with neural networks

Identifying Drivers of Predictive Aleatoric Uncertainty

Evaluating Explainability in Machine Learning Predictions through Explainer-Agnostic Metrics

Enabling Explainable Fusion in Deep Learning with Fuzzy Integral Neural Networks

Explainable data-driven modeling via mixture of experts: towards effective blending of grey and black-box models