iRoCo: Intuitive Robot Control From Anywhere Using a Smartwatch

Fabian C Weigend,Xiao Liu,Shubham Sonawani,Neelesh Kumar,Venugopal Vasudevan,Heni Ben Amor

2024-03-12

Abstract:This paper introduces iRoCo (intuitive Robot Control) - a framework for ubiquitous human-robot collaboration using a single smartwatch and smartphone. By integrating probabilistic differentiable filters, iRoCo optimizes a combination of precise robot control and unrestricted user movement from ubiquitous devices. We demonstrate and evaluate the effectiveness of iRoCo in practical teleoperation and drone piloting applications. Comparative analysis shows no significant difference between task performance with iRoCo and gold-standard control systems in teleoperation tasks. Additionally, iRoCo users complete drone piloting tasks 32\% faster than with a traditional remote control and report less frustration in a subjective load index questionnaire. Our findings strongly suggest that iRoCo is a promising new approach for intuitive robot control through smartwatches and smartphones from anywhere, at any time. The code is available at www.github.com/wearable-motion-capture

Robotics

What problem does this paper attempt to address?

The paper aims to address the challenges in current human-machine collaboration, particularly the reliance on expensive and limited motion capture systems (such as OptiTrack) for remote operation and drone control. To achieve more flexible and convenient human-machine interaction, the authors propose a new framework called iRoCo (Intuitive Robot Control). iRoCo optimizes the balance between precise robot control based on smartwatches and smartphones and user unconstrained mobility by integrating probabilistic differentiable filters. Specifically, the main objectives of this study include:

1. **Developing the iRoCo system**: Constructing a framework that enables intuitive robot control using only a smartwatch and a smartphone, allowing users to control robots anytime, anywhere.

2. **Utilizing differentiable filtering algorithms**: Employing differentiable recursive filtering techniques (especially the differentiable ensemble Kalman filter DEnKF) to improve the accuracy of human pose estimation and quantify uncertainty.

3. **Designing customized control modalities**: Developing a customized control method to enable users to perform remote operations through natural body movements, such as remote control via video streaming or intuitive drone control.

4. **Validating performance**: Evaluating the performance of the iRoCo system through practical applications (such as remote operation of pick-and-place tasks and drone flight tasks) and comparing it with existing gold standard control systems.

The research results show that iRoCo has comparable performance to traditional control methods in completing specific tasks, and even performs better in some cases. For example, in drone control tasks, users were able to complete tasks faster than using traditional remote controllers and reported lower frustration levels. This indicates that iRoCo is a promising new approach for achieving intuitive robot control from anywhere using smart devices.