Abstract:Advances in the performance of large language models (LLMs) have led some researchers to propose the emergence of theory of mind (ToM) in artificial intelligence (AI). LLMs can attribute beliefs, desires, intentions, and emotions, and they will improve in their accuracy. Rather than employing the characteristically human method of empathy, they learn to attribute mental states by recognizing linguistic patterns in a dataset that typically do not include that individual. We ask whether LLMs' inability to empathize precludes them from honoring an individual's right to be an exception, that is, from making assessments of character and predictions of behavior that reflect appropriate sensitivity to a person's individuality. Can LLMs seriously consider an individual's claim that their case is different based on internal mental states like beliefs, desires, and intentions, or are they limited to judging that case based on its similarities to others? We propose that the method of empathy has special significance for honoring the right to be an exception that is distinct from the value of predictive accuracy, at which LLMs excel. We conclude by considering whether using empathy to consider exceptional cases has intrinsic or merely practical value and we introduce conceptual and empirical avenues for advancing this investigation.

What problem does this paper attempt to address?

### Problems the Paper Attempts to Address This paper explores the capabilities and limitations of large language models (LLMs) in handling individuals' "exceptional rights." Specifically, the authors focus on: 1. **Whether LLMs can understand individuals' uniqueness through empathy like humans do**: Empathy refers to the attempt to understand others' motivations, personalities, decisions, and behaviors from their perspective. The authors argue that empathy is not only a method for understanding others' mental states but also a key to respecting individual uniqueness. They investigate whether LLMs possess this ability and whether the lack of this ability affects the fairness and accuracy of LLMs in evaluating individuals. 2. **Whether LLMs can consider individuals' particularities when evaluating them**: When individuals claim that their situations are unique and require special consideration, can LLMs take these claims seriously? Or do they merely judge based on similarities to other similar cases? The authors discuss the capabilities of LLMs in this regard and propose that LLMs' inability to understand individual uniqueness through empathy may affect the quality of their decisions in certain contexts. 3. **The importance of empathy in respecting individuals' "exceptional rights"**: The authors explore the unique value of empathy as a method in respecting individuals' "exceptional rights." They believe that although LLMs may excel in predicting behaviors and attributing mental states, they lack the crucial ability of empathy, which may affect their fair evaluation of individuals in certain contexts. ### Main Arguments - **Empathy and "Exceptional Rights"**: Empathy is not just the ability to understand others' feelings but also a method to respect individual uniqueness. The authors argue that respecting individuals' "exceptional rights" requires not only accurate predictions and evaluations but also a profound understanding and respect for individual uniqueness. - **Limitations of LLMs**: Although LLMs excel in predicting behaviors and attributing mental states, they lack the crucial ability of empathy. LLMs attribute mental states by recognizing language patterns rather than understanding individual uniqueness through empathy. - **Importance of Evaluation Methods**: The authors emphasize that the method of evaluating individuals affects not only the final judgment but also the respect for individuals. Even if LLMs can make accurate predictions, if they cannot fully consider individual uniqueness, these predictions may not be fair. ### Conclusion The authors believe that although LLMs have made significant progress in some areas, they still have limitations in respecting individuals' "exceptional rights." Empathy, as a unique evaluation method, has intrinsic value beyond its practical utility. Therefore, in contexts involving individual rights and dignity, human empathy remains irreplaceable.

Empathy and the Right to Be an Exception: What LLMs Can and Cannot Do

LLM Theory of Mind and Alignment: Opportunities and Risks

Can Large Language Models Exhibit Cognitive and Affective Empathy as Humans?

Large Language Models (LLMs) and Empathy - A Systematic Review

Do LLMs Exhibit Human-Like Reasoning? Evaluating Theory of Mind in LLMs for Open-Ended Responses

Empathic AI can't get under the skin

Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories?

LLMs grasp morality in concept

Exploring the psychology of LLMs' Moral and Legal Reasoning

Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBench

Rational Sensibility: LLM Enhanced Empathetic Response Generation Guided by Self-presentation Theory

Navigating LLM Ethics: Advancements, Challenges, and Future Directions

A Survey on Human-Centric LLMs

Modeling Emotions and Ethics with Large Language Models

LLMs and the Human Condition

Large Language Models are Capable of Offering Cognitive Reappraisal, if Guided

A Critical Reflection and Forward Perspective on Empathy and Natural Language Processing

The Impossibility of Fair LLMs

LLM-Based Empathetic Response Through Psychologist-Agent Debate

Are Large Language Models More Empathetic than Humans?