AsymDex: Leveraging Asymmetry and Relative Motion in Learning Bimanual Dexterity

Zhaodong Yang,Yunhai Han,Harish Ravichandar

2024-11-20

Abstract:We present Asymmetric Dexterity (AsymDex), a novel reinforcement learning (RL) framework that can efficiently learn asymmetric bimanual skills for multi-fingered hands without relying on demonstrations, which can be cumbersome to collect. Two crucial ingredients enable AsymDex to reduce the observation and action space dimensions and improve sample efficiency. First, AsymDex leverages the natural asymmetry found in human bimanual manipulation and assigns specific and interdependent roles to each hand: a facilitating hand that moves and reorients the object, and a dominant hand that performs complex manipulations on said object. Second, AsymDex defines and operates over relative observation and action spaces, facilitating responsive coordination between the two hands. Further, AsymDex can be easily integrated with recent advances in grasp learning to handle both the object acquisition phase and the interaction phase of bimanual dexterity. Unlike existing RL-based methods for bimanual dexterity, which are tailored to a specific task, AsymDex can be used to learn a wide variety of bimanual tasks that exhibit asymmetry. Detailed experiments on four simulated asymmetric bimanual dexterous manipulation tasks reveal that AsymDex consistently outperforms strong baselines that challenge its design choices, in terms of success rate and sample efficiency. The project website is at <a class="link-external link-https" href="https://sites.google.com/view/asymdex-2024/" rel="external noopener nofollow">this https URL</a>.

Robotics

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to use Reinforcement Learning (RL) to efficiently learn the asymmetric bimanual dexterous manipulation skills of multi - fingered hands without relying on demonstration data. Specifically, the researchers proposed a new framework - Asymmetric Dexterity (AsymDex) to address the following challenges:

1. **High - dimensional state and action spaces**: Bimanual dexterous manipulation involves high - dimensional state and action spaces, which makes the learning process complex and inefficient.
2. **Coordination and synchronization**: It is necessary to coordinate and synchronize the actions of the two arms, which is a huge challenge for traditional methods.
3. **Task diversity**: Existing RL methods are usually designed for specific tasks, while AsymDex aims to learn a variety of asymmetric bimanual tasks.

### Key elements of the solution

To overcome the above challenges, AsymDex introduced two key design ideas:

1. **Utilize natural asymmetry**:
- **Role assignment**: Divide the two hands into an "auxiliary hand" and a "dominant hand". The auxiliary hand is responsible for moving and re - positioning objects, while the dominant hand performs complex operations.
- **Simplify action space**: Since the auxiliary hand does not need to perform complex fingertip operations, its action space can be greatly simplified, only controlling the 6D pose of its base.

2. **Relative observation and action spaces**:
- **Relative coordinate system**: Define relative observation and action spaces to encourage responsive coordination between the two hands. By converting the observation and action spaces from the world coordinate system to the coordinate system relative to the object held by the auxiliary hand, unnecessary exploration is reduced and more complex coordinated behaviors are allowed.

### Experimental verification

The researchers evaluated AsymDex on four simulated asymmetric bimanual dexterous manipulation tasks, including:
- **Block in cup**: One hand puts a block into the cup held by the other hand.
- **Stack**: The two hands hold a cup respectively and stack them together.
- **Bottle cap**: One hand holds the bottle and re - orients it so that the other hand can unscrew the cap.
- **Switch**: One hand holds the switch and re - orients it so that the other hand can turn it on.

The experimental results show that AsymDex is significantly superior to other baseline methods in terms of success rate and sample efficiency, especially in complex bimanual coordination tasks.

### Summary

AsymDex effectively reduces the learning difficulty of bimanual dexterous manipulation and improves sample efficiency by introducing asymmetric role assignment and relative observation and action spaces. This framework provides new ideas and solutions for robots to perform complex bimanual tasks in the human environment.

### Formula display

The formulas involved in the paper are as follows:

1. **Relative position controller**:
\[
\hat{\xi}_b^d(t)=\alpha R_{of}^{world}\cdot\text{dist}(\hat{\xi}_b^r(t),\xi_b^r(t))+\xi_b^d(t)
\]
\[
\hat{\xi}_b^f(t)=(\alpha - 1)R_{of}^{world}\cdot\text{dist}(\hat{\xi}_b^r(t),\xi_b^r(t))+\xi_b^f(t)
\]
where \(R_{of}^{world}\) represents the rotation transformation from the object coordinate system \(P_f\) held by the auxiliary hand to the world coordinate system \(P_W\), \(\text{dist}(·)\) represents the difference between two 6D poses, and \(\alpha\) is a hyperparameter that controls the participation of each hand.

2. **Policy function**:
\[
\pi_{\text{AsymDex}}(\hat{\xi}_b^r(t),\hat{\xi}_h^d(t)|\xi_b^r(t),\xi_h^d(t),o_r(t))
\]

Through these formulas, AsymD