Motion Control of Interactive Robotic Arms Based on Mixed Reality Development

Hanxiao Chen

2024-01-03

Abstract:Mixed Reality (MR) is constantly evolving to inspire new patterns of robot manipulation for more advanced Human- Robot Interaction under the 4th Industrial Revolution Paradigm. Consider that Mixed Reality aims to connect physical and digital worlds to provide special immersive experiences, it is necessary to establish the information exchange platform and robot control systems within the developed MR scenarios. In this work, we mainly present multiple effective motion control methods applied on different interactive robotic arms (e.g., UR5, UR5e, myCobot) for the Unity-based development of MR applications, including GUI control panel, text input control panel, end-effector object dynamic tracking and ROS-Unity digital-twin connection.

Robotics,Human-Computer Interaction

What problem does this paper attempt to address?

The paper primarily addresses the issue of motion control for interactive robotic arms in a Mixed Reality (MR) environment. Specifically, the authors propose a Mixed Reality development framework based on the Unity engine, aiming to achieve effective motion control for different types of interactive robotic arms (such as UR5, UR5e, myCobot, etc.) through this framework.

To solve the aforementioned problem, the paper mainly undertakes the following work:

1. **Proposes two mainstream motion control methods**:
- “Control by Unity”: This method mainly utilizes the Unity 3D engine and its provided Graphical User Interface (GUI) and Inverse Kinematics (IK) modules to achieve intuitive and effective control of the robotic arm.
- “Control by ROS”: This method is based on the concept of Digital Twin, establishing a bridge between Unity and the Robot Operating System (ROS), and using the MoveIt! toolkit in ROS for motion planning, thereby achieving synchronous control of both virtual and real robotic arms.

2. **Provides a detailed introduction to inverse kinematics and motion planning technologies**: To support the above two control methods, the paper also delves into the basic principles and implementation methods of inverse kinematics, as well as related motion planning technologies, including traditional path planning and reinforcement learning-based motion planning strategies.

3. **Conducts experimental validation**: To verify the effectiveness of the proposed motion control methods, the authors conducted extensive experiments on different interactive robotic arms, including using GUI slider control panels, text input control panels, and Unity C# script control, among other methods to control the robotic arms, and demonstrated functionalities such as dynamic object tracking.

In summary, the goal of this paper is to provide efficient and intuitive motion control solutions for interactive robotic arms in a Mixed Reality environment, thereby promoting more advanced human-machine interaction applications.