Learning to Look Around: Enhancing Teleoperation and Learning with a Human-like Actuated Neck

Bipasha Sen,Michelle Wang,Nandini Thakur,Aditya Agarwal,Pulkit Agrawal

2024-11-02

Abstract:We introduce a teleoperation system that integrates a 5 DOF actuated neck, designed to replicate natural human head movements and perception. By enabling behaviors like peeking or tilting, the system provides operators with a more intuitive and comprehensive view of the environment, improving task performance, reducing cognitive load, and facilitating complex whole-body manipulation. We demonstrate the benefits of natural perception across seven challenging teleoperation tasks, showing how the actuated neck enhances the scope and efficiency of remote operation. Furthermore, we investigate its role in training autonomous policies through imitation learning. In three distinct tasks, the actuated neck supports better spatial awareness, reduces distribution shift, and enables adaptive task-specific adjustments compared to a static wide-angle camera.

Robotics

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to enhance the intuitiveness and efficiency of the tele - operation system by introducing an actuated neck with 5 degrees of freedom (5 - DOF), thereby improving the operator's task - performing ability and reducing cognitive load. Specifically, the system aims to:

1. **Improve the intuitiveness of tele - operation**: By imitating the natural head and neck movements of humans, operators can adjust the viewing angle more naturally, so as to better perceive and operate objects in complex environments.
2. **Improve data quality**: Use standard RGB cameras to provide high - quality images, reduce perception errors, and thus provide better data support for autonomous policy training.
3. **Enhance interactive perception**: By dynamically adjusting the camera angle, support object tracking and effective operation, imitating natural human behaviors.

The paper shows the performance of this tele - operation system with an actuated neck in seven challenging tele - operation tasks and explores its role in training autonomous policies through imitation learning. The experimental results show that, compared with the static wide - angle camera, the system with an actuated neck shows significant advantages in dealing with occlusion, adjusting the viewing angle, and adapting to different tasks.

### Formulas and Technical Details

- **Degrees of Freedom (DOF)**: The motion ability of the robot neck is represented by degrees of freedom. The formula is:
\[
\text{DOF} = 5
\]
This means that the neck can rotate and move on five different axes to simulate human head movements.

- **Task Success Rate**: In the experiment, the author compared the success rates of the system with an actuated neck and the static wide - angle camera system in different tasks. For example, in the "Cup from Bottom Shelf" (CfB) task, the success rate of the system with an actuated neck is 95%, while the success rate of the static wide - angle camera is 0%. The formulas are expressed as:
\[
\text{Success Rate}_{\text{CfB}}^{\text{Actuated}} = 95\%
\]
\[
\text{Success Rate}_{\text{CfB}}^{\text{Static}} = 0\%
\]

These improvements make the tele - operation system more efficient and intuitive, especially in tasks that require dealing with occlusion and complex environments.