Abstract:Background: Explainable artificial intelligence (XAI) is a technology that can enhance trust in mental state classifications by providing explanations for the reasoning behind artificial intelligence (AI) models outputs, especially for high-dimensional and highly-correlated brain signals. Feature importance and counterfactual explanations are two common approaches to generate these explanations, but both have drawbacks. While feature importance methods, such as shapley additive explanations (SHAP), can be computationally expensive and sensitive to feature correlation, counterfactual explanations only explain a single outcome instead of the entire model. Methods: To overcome these limitations, we propose a new procedure for computing global feature importance that involves aggregating local counterfactual explanations. This approach is specifically tailored to fMRI signals and is based on the hypothesis that instances close to the decision boundary and their counterfactuals mainly differ in the features identified as most important for the downstream classification task. We refer to this proposed feature importance measure as Boundary Crossing Solo Ratio (BoCSoR), since it quantifies the frequency with which a change in each feature in isolation leads to a change in classification outcome, i.e., the crossing of the model's decision boundary. Results and conclusions: Experimental results on synthetic data and real publicly available fMRI data from the Human Connect project show that the proposed BoCSoR measure is more robust to feature correlation and less computationally expensive than state-of-the-art methods. Additionally, it is equally effective in providing an explanation for the behavior of any AI model for brain signals. These properties are crucial for medical decision support systems, where many different features are often extracted from the same physiological measures and a gold standard is absent. Consequently, computing feature importance may become computationally expensive, and there may be a high probability of mutual correlation among features, leading to unreliable results from state-of-the-art XAI methods.

Better Model Selection with a new Definition of Feature Importance

Feature Importance versus Feature Influence and What It Signifies for Explainable AI

Sufficient and Necessary Explanations (and What Lies in Between)

Comparison of feature importance measures as explanations for classification models

Comparing interpretability and explainability for feature selection

Many Faces of Feature Importance: Comparing Built-in and Post-hoc Feature Importance in Text Classification

PRIME: Posterior reconstruction of the input for model explanations

From SHAP Scores to Feature Importance Scores

From unbiased MDI Feature Importance to Explainable AI for Trees

Feature selection strategies: a comparative analysis of SHAP-value and importance-based methods

A Guide to Feature Importance Methods for Scientific Inference

From local counterfactuals to global feature importance: efficient, robust, and model-agnostic explanations for brain connectivity networks

Exclusion and Inclusion -- A model agnostic approach to feature importance in DNNs

Inherent Inconsistencies of Feature Importance

The Analysis and Development of an XAI Process on Feature Contribution Explanation.

Leveraging Black-box Models to Assess Feature Importance in Unconditional Distribution

Importance estimate of features via analysis of their weight and gradient profile

Confident Feature Ranking

Selective Explanations

A review and benchmark of feature importance methods for neural networks

Factor Importance Ranking and Selection using Total Indices