Abstract:The rationale behind a deep learning model's output is often difficult to understand by humans. EXplainable AI (XAI) aims at solving this by developing methods that improve interpretability and explainability of machine learning models. Reliable evaluation metrics are needed to assess and compare different XAI methods. We propose a novel evaluation approach for benchmarking state-of-the-art XAI attribution methods. Our proposal consists of a synthetic classification model accompanied by its derived ground truth explanations allowing high precision representation of input nodes contributions. We also propose new high-fidelity metrics to quantify the difference between explanations of the investigated XAI method and those derived from the synthetic model. Our metrics allow assessment of explanations in terms of precision and recall separately. Also, we propose metrics to independently evaluate negative or positive contributions of inputs. Our proposal provides deeper insights into XAI methods output. We investigate our proposal by constructing a synthetic convolutional image classification model and benchmarking several widely used XAI attribution methods using our evaluation approach. We compare our results with established prior XAI evaluation metrics. By deriving the ground truth directly from the constructed model in our method, we ensure the absence of bias, e.g., subjective either based on the training set. Our experimental results provide novel insights into the performance of Guided-Backprop and Smoothgrad XAI methods that are widely in use. Both have good precision and recall scores among positively contributing pixels (0.7, 0.76 and 0.7, 0.77, respectively), but poor precision scores among negatively contributing pixels (0.44, 0.61 and 0.47, 0.75, resp.). The recall scores in the latter case remain close. We show that our metrics are among the fastest in terms of execution time.

Evaluating explainable artificial intelligence methods for multi-label deep learning classification tasks in remote sensing

Quantitative Analysis of Primary Attribution Explainable Artificial Intelligence Methods for Remote Sensing Image Classification

Extending CAM-based XAI methods for Remote Sensing Imagery Segmentation

Opening the Black Box: A systematic review on explainable artificial intelligence in remote sensing

Explainable Analysis of Deep Learning Methods for SAR Image Classification

Explainable AI for Earth Observation: Current Methods, Open Challenges, and Opportunities

Trainable Noise Model as an XAI evaluation method: application on Sobol for remote sensing image segmentation

AI explainability framework for environmental management research

Opening the Black-Box: A Systematic Review on Explainable AI in Remote Sensing

Towards transparency in AI: Explainable bird species image classification for ecological research

Explainable GeoAI: can saliency maps help interpret artificial intelligence's learning process? An empirical study on natural feature detection

Underwater SONAR Image Classification and Analysis using LIME-based Explainable Artificial Intelligence

XC: Exploring Quantitative Use Cases for Explanations in 3D Object Detection

EXplainable Artificial Intelligence (XAI)—From Theory to Methods and Applications

Unlocking the Potential of Explainable Artificial Intelligence in Remote Sensing Big Data

Analyzing and assessing explainable AI models for smart agriculture environments

Interpretable Geoscience Artificial Intelligence (XGeoS-AI): Application to Demystify Image Recognition

Precise Benchmarking of Explainable AI Attribution Methods

Classification Metrics for Image Explanations: Towards Building Reliable XAI-Evaluations

Interpretability is in the Eye of the Beholder: Human Versus Artificial Classification of Image Segments Generated by Humans Versus XAI

Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey