Reactions to Immoral AI Decisions: the Moral Deficit Effect and Its Underlying Mechanism
Xiaoyong Hu,Mufeng Li,Dixin Wang,Feng Yu
DOI: https://doi.org/10.1360/tb-2023-1094
2024-01-01
Abstract:Artificial intelligence (AI) is a branch of computer science in which systems are created with intelligence that enables them to perform cognitive functions, such as perception, reasoning, learning, and decision-making. AI has formidable problem-solving abilities in various domains, such as surveillance, health care, and finance. However, its societal applications raise ethical issues, such as gender and racial discrimination. Moreover, psychological research on AI ethics has revealed that people react less morally to unethical decisions made by AI than to those made by humans, showing deficiencies in moral evaluations, punishments, and behavioral responses to AI. These moral deficits of AI decisions have serious negative impacts on individuals, organizations, and society. For example, research has confirmed that this effect leads companies to use AI in their recruitment processes to discriminate against job applicants, reduces people's awareness of unethical behavior, enables companies to evade accountability, exacerbates the challenges of advocacy and justice for affected groups, and even undermines societal moral standards along with the widespread use of AI. These negative impacts will damage trust in justice and fairness, causing lasting harm to societal ethics. How can we explain AI moral deficit effects? We draw on the mind perception theory, which suggests that people's moral reactions depend on how they perceive the mental attributes of an entity. These attributes can be divided into two dimensions: agency and experience. Agency refers to the belief that the entity can act, plan, self-regulate, remember, communicate, and think like a typical adult human. Experience refers to the belief that the entity can feel emotional states, such as hunger, fear, and pain. Mind perception is the process of attributing mental capabilities to an entity, while moral judgment is the process of evaluating the entity as good or bad. Entities with high agency are held responsible for their moral actions, and entities with high experience are expected to behave safely and ethically. People tend to view AI as having less agency and experience than adults do, which leads to the AI moral deficit effect. We explore the psychological reasons behind people's lack of moral concern about immoral decisions made by AI. We contrast people's moral responses to AI and human decision-makers and the serious social implications of this difference. Based on the theory of mind perception and moral dualism and supported by empirical evidence, we claim that two parallel factors underlie the moral deficit effect of AI decision-making: agency and experience. Additionally, we show that anthropomorphism and expectation violation can influence this effect. Thus, we propose a theoretical model of the inherent psychological mechanism of the moral deficit effect of AI decision-making from the perspective of a moral agent. This model extends the theory of mind perception and deepens the understanding of moral dualism. In sum, we investigate how people react differently to biased AI and biased humans. Unlike other disciplines (e.g., computer science, philosophy, law, sociology) that focus on the design of fair algorithms in their "algorithmic ethics" research, we examine the human side of the problem. We contend that comprehending these differences is crucial for tackling the social challenges of biased AI and proposing a novel approach to building fair algorithms, as well as a fresh outlook on "algorithmic ethics," which shifts the attention from algorithmic design to human responses. This approach is vital for ethics and for AI researchers to devise ethical frameworks and guidelines for AI systems. Furthermore, the findings can guide policy-making, legal frameworks and social interventions to reduce the adverse effects of AI and foster social equity and justice. However, due to the dynamic and fast-changing nature of AI, this field still needs additional research on the underlying mechanisms and intervention strategies involved.