Abstract:We focus on the task of goal-oriented grasping, in which a robot is supposed to grasp a pre-assigned goal object in clutter and needs some pre-grasp actions such as pushes to enable stable grasps. However, in this task, the robot gets positive rewards from environment only when successfully grasping the goal object. Besides, joint pushing and grasping elongates the action sequence, compounding the problem of reward delay. Thus, sample inefficiency remains a main challenge in this task. In this paper, a goal-conditioned hierarchical reinforcement learning formulation with high sample efficiency is proposed to learn a push-grasping policy for grasping a specific object in clutter. In our work, sample efficiency is improved by two means. First, we use a goal-conditioned mechanism by goal relabeling to enrich the replay buffer. Second, the pushing and grasping policies are respectively regarded as a generator and a discriminator and the pushing policy is trained with supervision of the grasping discriminator, thus densifying pushing rewards. To deal with the problem of distribution mismatch caused by different training settings of two policies, an alternating training stage is added to learn pushing and grasping in turn. A series of experiments carried out in simulation and real world indicate that our method can quickly learn effective pushing and grasping policies and outperforms existing methods in task completion rate and goal grasp success rate by less times of motion. Furthermore, we validate that our system can also adapt to goal-agnostic conditions with better performance. Note that our system can be transferred to the real world without any fine-tuning. Our code is available at <a class="link-external link-https" href="https://github.com/xukechun/Efficient_goal-oriented_push-grasping_synergy" rel="external noopener nofollow">this https URL</a>.

Progressively Learning to Reach Remote Goals by Continuously Updating Boundary Goals

Learning Hierarchical Graph-Based Policy for Goal-Reaching in Unknown Environments

Goal-Reaching Policy Learning from Non-Expert Observations via Effective Subgoal Guidance

Goal-Conditioned Reinforcement Learning with Disentanglement-based Reachability Planning

Solving Robotic Manipulation With Sparse Reward Reinforcement Learning Via Graph-Based Diversity and Proximity

Learning of Long-Horizon Sparse-Reward Robotic Manipulator Tasks With Base Controllers

Active Hierarchical Exploration with Stable Subgoal Representation Learning

Goal exploration augmentation via pre-trained skills for sparse-reward long-horizon goal-conditioned reinforcement learning

Reinforcement Learning with Goal-Distance Gradient

Bidirectional-Reachable Hierarchical Reinforcement Learning with Mutually Responsive Policies

Efficient Hierarchical Exploration with an Active Subgoal Generation Strategy.

Revisiting Sparse Rewards for Goal-Reaching Reinforcement Learning

Keeping Your Distance: Solving Sparse Reward Tasks Using Self-Balancing Shaped Rewards

Breadcrumbs to the Goal: Goal-Conditioned Exploration from Human-in-the-Loop Feedback

Offline Goal-Conditioned Reinforcement Learning for Safety-Critical Tasks with Recovery Policy

Computational Benefits of Intermediate Rewards for Goal-Reaching Policy Learning

Efficient learning of goal-oriented push-grasping synergy in clutter

TLDR: Unsupervised Goal-Conditioned RL via Temporal Distance-Aware Representations

Robot Subgoal-guided Navigation in Dynamic Crowded Environments with Hierarchical Deep Reinforcement Learning

Relay Hindsight Experience Replay: Self-guided continual reinforcement learning for sequential object manipulation tasks with sparse rewards

Robotic Manipulation in Dynamic Scenarios via Bounding-Box-Based Hindsight Goal Generation