Abstract:We argue that the trend toward providing users with feasible and actionable explanations of AI decisions, known as recourse explanations, comes with ethical downsides. Specifically, we argue that recourse explanations face several conceptual pitfalls and can lead to problematic explanation hacking, which undermines their ethical status. As an alternative, we advocate that explanations of AI decisions should aim at understanding.
What problem does this paper attempt to address?
The core problem that this paper attempts to solve is the ethical dilemma in algorithmic explanation, especially the potential problems regarding "algorithmic recourse". Specifically, the author believes that providing users with feasible and actionable explanations of AI decisions (i.e., "algorithmic recourse") may seem reasonable on the surface, but in fact, it may bring a series of conceptual traps and ethical problems, which may undermine its ethical status.
### Main Problems
1. **Responsibility Transfer**:
- Algorithmic recourse may transfer the responsibility from the AI system or institution to the end - user or data subject. For example, in the case of loan rejection, if the AI suggests that the user obtain approval by changing their financial situation, this may make the user feel that the problem lies with themselves, rather than the fairness and robustness of the bank or the AI system.
2. **Hidden Assumptions**:
- Algorithmic recourse is usually based on some implicit assumptions, which may be overly simplified or ignore the complex real - life situations of users. For example, suggesting that the user change their occupation or reduce their debt without considering factors such as social structure, economic conditions, and personal background, making these suggestions difficult to implement in practice.
3. **Explanation Hacking**:
- Users may use algorithmic recourse to carry out "explanation hacking", that is, change the input features in a specific way to obtain favorable AI decisions, and these ways may not conform to the actual situation or ethical standards. This behavior will further undermine the ethical value of algorithmic recourse.
### The Author's Claim
The author advocates a "understanding - first" view, that is, the explanation of AI decisions should aim to help users understand why the AI made a certain decision, rather than simply providing feasible operational suggestions. The "understanding - first" view can avoid the above problems and better meet users' needs for transparency and fairness.
### Formula Representation
No specific mathematical formulas are involved in the discussion, but if it is necessary to reference relevant concepts, the following Markdown format can be used:
- **Responsibility Transfer Formula**: $R_{\text{user}} = R_{\text{system}} + \Delta R$
- **Hidden Assumptions Formula**: $\mathbb{E}[A|B] = \sum_{i} P(A_i | B) \cdot A_i$
- **Explanation Hacking Formula**: $H = f(\text{CE}) + g(\text{User Action})$
Here, $R_{\text{user}}$ represents the user's responsibility, $R_{\text{system}}$ represents the system's responsibility, $\Delta R$ represents the amount of responsibility transfer; $\mathbb{E}[A|B]$ represents the expected value of $A$ under the given condition $B$; $H$ represents the possibility of hacking, and $f(\text{CE})$ and $g(\text{User Action})$ represent the influence functions of the explanation and user behavior respectively.
In this way, the author hopes to guide readers to rethink the goals of AI explanation, emphasize the importance of understanding, and point out the potential problems in the currently popular algorithmic recourse methods.