Abstract:Emergent behavior arising in a joint human-robot system cannot be fully predicted based on an understanding of the individual agents. Typically, robot behavior is governed by algorithms that optimize a reward function that should quantitatively capture the joint system's goal. Although reward functions can be updated to better match human needs, this is no guarantee that no misalignment with the complex and variable human needs will occur. Algorithms may learn undesirable behavior when interacting with the human and the intrinsically unpredictable human-inhabited world, thereby producing further misalignment with human users or bystanders. As a result, humans might behave differently than anticipated, causing robots to learn differently and undesirable behavior to emerge. With this short paper, we state that to design for Human-Robot Interaction that mitigates such undesirable emergent behavior, we need to complement advancements in human-robot interaction algorithms with human factors knowledge and expertise. More specifically, we advocate a three-pronged approach that we illustrate using a particularly challenging example of safety-critical human-robot interaction: a driver interacting with a semi-automated vehicle. Undesirable emergent behavior should be mitigated by a combination of 1) including driver behavioral mechanisms in the vehicle's algorithms and reward functions, 2) model-based approaches that account for interaction-induced driver behavioral adaptations and 3) driver-centered interaction design that promotes driver engagement with the semi-automated vehicle, and the transparent communication of each agent's actions that allows mutual support and adaptation. We provide examples from recent empirical work in our group, in the hope this proves to be fruitful for discussing emergent human-robot interaction.

Safety Issues in Human-Machine Collaboration and Possible Countermeasures

A Review of Human-Machine Cooperation in the Robotics Domain

Towards Safe Multi-Level Human-Robot Interaction in Industrial Tasks

Challenges of human—machine collaboration in risky decision-making

Toward human-vehicle collaboration: Review and perspectives on human-centered collaborative automated driving

Human-Computer Interaction and Human-AI Collaboration in Advanced Air Mobility: A Comprehensive Review

Need for Developing a Security Robot-Based Risk Management for Emerging Practices in the Workplace Using the Advanced Human-Robot Collaboration Model

Mitigating undesirable emergent behavior arising between driver and semi-automated vehicle

Safety-aware human-centric collaborative assembly

Adaptive Safety-critical Control with Uncertainty Estimation for Human-robot Collaboration

Symbiotic Safety: Safe and Efficient Human-Machine Collaboration by utilizing Rules

Engineering Human–Machine Teams for Trusted Collaboration

Taking Charge of One's Own Safety While Collaborating with Robots: Enhancing Situational Awareness for a Safe Environment

A framework and method for Human-Robot cooperative safe control based on digital twin

Human-Machine Collaboration for Smart Decision Making: Current Trends and Future Opportunities

Human-robot collaboration assembly personalised safety detection method based on digital twin

Safeguarding AI Agents: Developing and Analyzing Safety Architectures

Autonomous Heavy-Duty Mobile Machinery: A Multidisciplinary Collaborative Challenge

A Complementary Framework for Human-Robot Collaboration with a Mixed AR-Haptic Interface

A collaborative intelligence-based approach for handling human-robot collaboration uncertainties

Reflections on Automation Complacency