Abstract:A flexible production model in which humans and robots collaborate has become an urgent need for industrial production. Therefore, research on human-machine integration systems has received more and more attention in recent years. Human-computer interaction is widely used in industrial production and entertainment life, such as industrial control, VR games, etc. In this article, we designed and implemented a human-machine fusion system that meets different interactive tasks. Human movement estimation and robot interactive control are the key technologies. First, we propose a human posture solution method based on information fusion of multiple cameras and inertial measurement units. This method constructs an optimization problem, fuses the 2D joint detection information under multiple cameras and the measurement information of the worn inertial measurement unit, and optimizes the estimation of human kinematic posture, which improves the problems of incomplete posture information and sensitivity to noise under a single sensor. Therefore, this method improves the accuracy of pose estimation. Secondly, combining the kinematic characteristics of the robot and the characteristics of human-computer interaction, we designed a robot control strategy based on target point tracking and model predictive control, so that the robot can adapt to dynamic environments and different interaction needs by adjusting control parameters. At the same time, our design ensures the safety of robots and operators. Finally, we conducted human-computer interaction experiments such as action following, item delivery, and active obstacle avoidance. The experimental results show the effectiveness and reliability of the designed robot interaction system in a human-machine fusion environment.

A Universal Self-Adaption Workspace Mapping Method for Human–Robot Interaction Using Kinect Sensor Data

The Design and Implementation of Natural Human-Robot Interaction System Based on Kinect Sensor

Design of Kinect-Based Human Robot Interaction Systems for A Robocup Middle Size League Soccer Robot

A Haptic Shared Control Architecture for Tracking of a Moving Object.

A Human-Robot Collaboration System for Object Handover

Simulation of One Effective Human-Robot Cooperation Method Based on Kinect Sensor and Uncalibrated Visual Servoing

One New Human-Robot Cooperation Method Based on Kinect Sensor and Visual-Servoing.

A Complementary Framework for Human-Robot Collaboration with a Mixed AR-Haptic Interface

Human–robot Kinematic Mapping Method Based on Index Constraint

An Augmented Discrete-Time Approach for Human-Robot Collaboration

A Multi-sensor Gesture Interaction System for Human-robot Cooperation

Human-Robot Interaction Control By Using A General Sensory Model

Human-robot cooperation using Kinect

Implementation of Engagement Detection for Human–Robot Interaction in Complex Environments

An Ergo-Interactive Framework for Human-Robot Collaboration Via Learning from Demonstration

Design of Human-Machine Integration System to Meet Diverse Interactive Tasks

Human-Manipulator Interface Using Hybrid Sensors Via CMAC for Dual Robots.

Human-robot Kinematics Mapping Method Based on Dynamic Equivalent Points

Markerless human-robot interface for dual robot manipulators using Kinect sensor

Proactive Robot Control for Collaborative Manipulation Using Human Intent

Human-robot Shared Control for Humanoid Manipulator Trajectory Planning