Abstract:Humanoid robots are equipped with humanoid arms to make them more acceptable to the general public. Humanoid robots are a great challenge in robotics. The concept of digital twin technology complies with the guiding ideology of not only Industry 4.0, but also Made in China 2025. This paper proposes a scheme that combines deep reinforcement learning (DRL) with digital twin technology for controlling humanoid robot arms. For rapid and stable motion planning for humanoid robots, multitasking-oriented training using the twin synchro-control (TSC) scheme with DRL is proposed. For switching between tasks, the robot arm training must be quick and diverse. In this work, an approach for obtaining a priori knowledge as input to DRL is developed and verified using simulations. Two simple examples are developed in a simulation environment. We developed a data acquisition system to generate angle data efficiently and automatically. These data are used to improve the reward function of the deep deterministic policy gradient (DDPG) and quickly train the robot for a task. The approach is applied to a model of the humanoid robot BHR-6, a humanoid robot with multiple-motion mode and a sophisticated mechanical structure. Using the policies trained in the simulations, the humanoid robot can perform tasks that are not possible to train with existing methods. The training is fast and allows the robot to perform multiple tasks. Our approach utilizes human joint angle data collected by the data acquisition system to solve the problem of a sparse reward in DRL for two simple tasks. A comparison with simulation results for controllers trained using the vanilla DDPG show that the designed controller developed using the DDPG with the TSC scheme have great advantages in terms of learning stability and convergence speed.

Neural-Dynamic-Method-Based Dual-Arm CMG Scheme with Time-Varying Constraints Applied to Humanoid Robots.

Dynamic Movement Primitive Based Motion Retargeting for Dual-Arm Sign Language Motions

A Time-Varying-Constrained Motion Generation Scheme for Humanoid Robot Arms.

Dynamical Obstacle Avoidance of Task- Constrained Mobile Manipulation Using Model Predictive Control

Balance maintenance in high-speed motion of humanoid robot arm-based on the 6D constraints of momentum change rate.

Cyclic motion generation of multi-link planar robot performing square end-effector trajectory analyzed via gradient-descent and Zhang et al’s neural-dynamic methods

A Multitasking-Oriented Robot Arm Motion Planning Scheme Based on Deep Reinforcement Learning and Twin Synchro-Control

An adaptive discretized RNN algorithm for posture collaboration motion control of constrained dual-arm robots

Acceleration-Level Cyclic-Motion Generation of Constrained Redundant Robots Tracking Different Paths

Neural-Dynamic Based Synchronous-Optimization Scheme of Dual Redundant Robot Manipulators

Motion Planning for Dual-Arm Robot Based on Soft Actor-Critic

Modification of Gesture-Determined-Dynamic Function with Consideration of Margins for Motion Planning of Humanoid Robots

Mutual-Collision-Avoidance Scheme Synthesized by Neural Networks for Dual Redundant Robot Manipulators Executing Cooperative Tasks

Bio-inspired Design and Inverse Kinematics Solution of an Omnidirectional Humanoid Robotic Arm with Geometric and Load Capacity Constraints

Two model-free schemes for solving kinematic tracking control of redundant manipulators using CMAC networks

CBMC: A Biomimetic Approach for Control of a 7-Degree of Freedom Robotic Arm

A Globally Guided Dual-Arm Reactive Motion Controller for Coordinated Self-Handover in a Confined Domestic Environment

Human-Manipulator Interface Using Hybrid Sensors Via CMAC for Dual Robots.

A Dual-Arm Robot Cooperation Framework Based on a Nonlinear Model Predictive Cooperative Control