Abstract:Predicting human motion based on a sequence of past observations is crucial for various applications in robotics and computer vision. Currently, this problem is typically addressed by training deep learning models using some of the most well-known 3D human motion datasets widely used in the community. However, these datasets generally do not consider how humans behave and move when a robot is nearby, leading to a data distribution different from the real distribution of motion that robots will encounter when collaborating with humans. Additionally, incorporating contextual information related to the interactive task between the human and the robot, as well as information on the human willingness to collaborate with the robot, can improve not only the accuracy of the predicted sequence but also serve as a useful tool for robots to navigate through collaborative tasks successfully. In this research, we propose a deep learning architecture that predicts both 3D human body motion and human intention for collaborative tasks. The model employs a multi-head attention mechanism, taking into account human motion and task context as inputs. The resulting outputs include the predicted motion of the human body and the inferred human intention. We have validated this architecture in two different tasks: collaborative object handover and collaborative grape harvesting. While the architecture remains the same for both tasks, the inputs differ. In the handover task, the architecture considers human motion, robot end effector, and obstacle positions as inputs. Additionally, the model can be conditioned on the desired intention to tailor the output motion accordingly. To assess the performance of the collaborative handover task, we conducted a user study to evaluate human perception of the robot's sociability, naturalness, security, and comfort. This evaluation was conducted by comparing the robot's behavior when it utilized the prediction in its planner versus when it did not. Furthermore, we also applied the model to a collaborative grape harvesting task. By integrating human motion prediction and human intention inference, our architecture shows promising results in enhancing the capabilities of robots in collaborative scenarios. The model's flexibility allows it to handle various tasks with different inputs, making it adaptable to real-world applications.

Context Aware Robot Architecture, Application to the RoboCup@Home Challenge

Setting Up Pepper For Autonomous Navigation And Personalized Interaction With Users

Context-aware Sound Event Recognition for Home Service Robots.

Interaction and Autonomy in RoboCup@Home and Building-Wide Intelligence

Hold My Hand: Development of a Force Controller and System Architecture for Joint Walking with a Companion Robot

Perception-enhancement Based Task Learning and Action Scheduling for Robotic Limb in CPS Environment

An Embedded Framework for Fully Autonomous Object Manipulation in Robotic-Empowered Assisted Living

RoboCup@Home Education 2020 Best Performance: RoboBreizh, a modular approach

Upgrading Pepper Robot s Social Interaction with Advanced Hardware and Perception Enhancements

Perspectives on usability and accessibility of an autonomous humanoid robot living with elderly people

A Hierarchical Architecture for Human-Robot Cooperation Processes

Enabling a Pepper Robot to provide Automated and Interactive Tours of a Robotics Laboratory

A Modular Framework for Flexible Planning in Human-Robot Collaboration

DAC-h3: A Proactive Robot Cognitive Architecture to Acquire and Express Knowledge About the World and the Self

HIPer: A Human-Inspired Scene Perception Model for Multifunctional Mobile Robots

Enhancing Robotic Collaborative Tasks Through Contextual Human Motion Prediction and Intention Inference

CASPER: Cognitive Architecture for Social Perception and Engagement in Robots

LAAIR: A Layered Architecture for Autonomous Interactive Robots

ARMCHAIR: integrated inverse reinforcement learning and model predictive control for human-robot collaboration

Some essential skills and their combination in an architecture for a cognitive and interactive robot

MaestROB: A Robotics Framework for Integrated Orchestration of Low-Level Control and High-Level Reasoning