Abstract:Attribution-based explanations are garnering increasing attention recently and have emerged as the predominant approach towards \textit{eXplanable Artificial Intelligence}~(XAI). However, the absence of consistent configurations and systematic investigations in prior literature impedes comprehensive evaluations of existing methodologies. In this work, we introduce {Meta-Rank}, an open platform for benchmarking attribution methods in the image domain. Presently, Meta-Rank assesses eight exemplary attribution methods using six renowned model architectures on four diverse datasets, employing both the \textit{Most Relevant First} (MoRF) and \textit{Least Relevant First} (LeRF) evaluation protocols. Through extensive experimentation, our benchmark reveals three insights in attribution evaluation endeavors: 1) evaluating attribution methods under disparate settings can yield divergent performance rankings; 2) although inconsistent across numerous cases, the performance rankings exhibit remarkable consistency across distinct checkpoints along the same training trajectory; 3) prior attempts at consistent evaluation fare no better than baselines when extended to more heterogeneous models and datasets. Our findings underscore the necessity for future research in this domain to conduct rigorous evaluations encompassing a broader range of models and datasets, and to reassess the assumptions underlying the empirical success of different attribution methods. Our code is publicly available at \url{<a class="link-external link-https" href="https://github.com/TreeThree-R/Meta-Rank" rel="external noopener nofollow">this https URL</a>}.

Causes and reasons in attribution theory: A conceptual critique.

How People Explain Behavior: A New Theoretical Framework

Attribution theory: An introduction to the special issue

A cognitive (attribution)-emotion-action model of motivated behavior: An analysis of judgments of help-giving.

A behavior analytic interpretation of attributions in the context of behavior therapy

Is causation deterministic or probabilistic? A critique of Frosch and Johnson-Laird (2011)

Causes and Explanations: A Structural-Model Approach --- Part 1: Causes

The Cause-Effect Problem: Motivation, Ideas, and Popular Misconceptions

Culture and Cause: American and Chinese Attributions for Social and Physical Events.

Pity, Anger, and Guilt

Effects of Causes and Causes of Effects

The role of causal attributions in determining behavioral consequences: A meta-analysis from an intrapersonal attributional perspective in achievement contexts.

A critique of motivation constructs to explain higher-order behavior: We should unpack the black box

What happens why? Psychological distance and focusing on causes versus consequences of events.

Linkages Between Causal Ascriptions, Emotion, and Behaviour

Analytic Criminology: Mechanisms and Methods in the Explanation of Crime and its Causes

Culpable Control and Counterfactual Reasoning in the Psychology of Blame

Effects of Manipulation on Attributions of Causation, Free Will, and Moral Responsibility

Culture and the Attribution Process and Probability Models:An Exploration

On the Evaluation Consistency of Attribution-based Explanations

Two conceptions of reason