Abstract:Machine ethics is the field that studies how ethical behaviour can be accomplished by autonomous systems. While there exist some systematic reviews aiming to consolidate the state of the art in machine ethics prior to 2020, these tend to not include work that uses reinforcement learning agents as entities whose ethical behaviour is to be achieved. The reason for this is that only in the last years we have witnessed an increase in machine ethics studies within reinforcement learning. We present here a systematic review of reinforcement learning for machine ethics and machine ethics within reinforcement learning. Additionally, we highlight trends in terms of ethics specifications, components and frameworks of reinforcement learning, and environments used to result in ethical behaviour. Our systematic review aims to consolidate the work in machine ethics and reinforcement learning thus completing the gap in the state of the art machine ethics landscape

What problem does this paper attempt to address?

The main aim of this paper is to address the issue of how to achieve machine ethics in Reinforcement Learning (RL). Specifically, the authors focus on how to enable RL agents to exhibit ethical behavior during the decision-making process. The key contributions of the paper can be summarized as follows: 1. **Systematic Review**: The authors conducted a systematic literature review to integrate the latest research findings in the fields of reinforcement learning and machine ethics. This work fills a gap in existing reviews, which typically only cover progress up to 2020 and do not sufficiently address the use of RL agents to achieve ethical behavior. 2. **Research Trend Analysis**: Through the analysis of relevant literature, the authors identified several key trends, including: - **Multi-Objective Reinforcement Learning**: Considering ethical norms as one of the additional objectives. - **Constrained Reinforcement Learning**: Viewing ethical behavior as a constraint on the agent's actions. - **Safe Reinforcement Learning**: Ensuring that the agent maintains ethical behavior throughout the training process. - **Multi-Agent Reinforcement Learning**: Achieving ethical behavior in multi-agent environments by maximizing collective benefits. 3. **Application of Ethical Theories**: Although most studies do not explicitly state the specific ethical theories adopted, the authors categorized the research content into several types, mainly consequentialism, deontology, virtue ethics, and approaches based on human expert values. 4. **Implementation Cases**: Researchers tested their methods in various environments, including fair resource allocation, abstract environments with prohibited or preferred states, motion planning scenarios with conflicting goals and values, complex simulated environments, and other specific cases. 5. **Types of Research**: The authors noted the diversity in research types, including theoretical/argumentative, empirical, and mathematical proofs. This reflects a trend in the field transitioning from theoretical exploration to empirical research. 6. **Human Factors**: For the sources of ethical norms, researchers adopted different approaches, including decisions by designers/developers, user customization, cooperative human participants, and demonstrations by human experts. The authors emphasized the importance of clearly stating ethical assumptions and pointed out that diverse perspectives should be considered to reduce the impact of bias. In summary, this paper systematically reviews and analyzes recent research findings on reinforcement learning and machine ethics, aiming to advance the field, particularly in the encoding and implementation of ethical norms.

Reinforcement Learning and Machine ethics:a systematic review

Learning Machine Morality through Experience and Interaction

Reinforcement Learning Review: Past Acts, Present Facts and Future Prospects

Landscape of Machine Implemented Ethics

Can Machine Learning be Moral?

A Word on Machine Ethics: A Response to Jiang et al. (2021)

Evolutionary Reinforcement Learning: A Systematic Review and Future Directions

Modeling Moral Choices in Social Dilemmas with Multi-Agent Reinforcement Learning

A Conceptual Framework for Ethical Evaluation of Machine Learning Systems

Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories?

Reinforcement Learning in Game Industry—Review, Prospects and Challenges

Reinforcement Learning in Practice: Opportunities and Challenges

Machine Ethics: The Creation of a Virtuous Machine

Do Artificial Reinforcement-Learning Agents Matter Morally?

Ethics of AI: A Systematic Literature Review of Principles and Challenges

Reinforcement Learning Algorithms and Applications in Healthcare and Robotics: A Comprehensive and Systematic Review

Reinforcement Learning Algorithms: An Overview and Classification

Ethics and Governance of Artificial Intelligence: A Survey of Machine Learning Researchers (extended Abstract)

Autonomous Industrial Management via Reinforcement Learning: Self-Learning Agents for Decision-Making -- A Review

Reinforcement Learning in Robotics: Challenges and Applications

The Challenges of Machine Learning: A Critical Review