Abstract:Robotic grasping in densely cluttered environments is challenging due to scarce collision-free grasp affordances. Non-prehensile actions can increase feasible grasps in cluttered environments, but most research focuses on single-arm rather than dual-arm manipulation. Policies from single-arm systems fail to fully leverage the advantages of dual-arm coordination. We propose a target-oriented hierarchical deep reinforcement learning (DRL) framework that learns dual-arm push-grasp synergy for grasping objects to enhance dexterous manipulation in dense clutter. Our framework maps visual observations to actions via a pre-trained deep learning backbone and a novel CNN-based DRL model, trained with Proximal Policy Optimization (PPO), to develop a dual-arm push-grasp strategy. The backbone enhances feature mapping in densely cluttered environments. A novel fuzzy-based reward function is introduced to accelerate efficient strategy learning. Our system is developed and trained in Isaac Gym and then tested in simulations and on a real robot. Experimental results show that our framework effectively maps visual data to dual push-grasp motions, enabling the dual-arm system to grasp target objects in complex environments. Compared to other methods, our approach generates 6-DoF grasp candidates and enables dual-arm push actions, mimicking human behavior. Results show that our method efficiently completes tasks in densely cluttered environments. <a class="link-external link-https" href="https://sites.google.com/view/pg4da/home" rel="external noopener nofollow">this https URL</a>

What problem does this paper attempt to address?

This paper attempts to solve the challenging problem of robots grasping objects in dense and cluttered environments. Specifically, its main objective is to enhance the dexterous manipulation ability of robots in complex environments through the coordinated pushing and grasping actions of two arms. The following are the specific problems proposed in the paper and their solutions: ### Main problems: 1. **Sparse collision - free grasping opportunities**: In dense and cluttered environments, it is very difficult to find collision - free grasping points. 2. **Limitations of single - arm systems**: Most research focuses on single - arm manipulation, but single - arm systems are often insufficient to complete tasks in dense and cluttered environments. 3. **Lack of two - arm coordination strategies**: Existing methods fail to fully utilize the advantages of two - arm coordination. ### Solutions: To solve the above problems, the authors propose a goal - oriented hierarchical framework based on deep reinforcement learning (DRL), which can learn two - arm pushing - grasping coordination strategies to improve the grasping success rate in dense and cluttered environments. Specific improvements include: 1. **Adaptive pushing action generation**: Flexible two - arm pushing actions are achieved by sampling from the learned feature maps, projecting the connected pixels into 3D space, and smoothing the 3D trajectories using the Savitzky - Golay filter. 2. **6 - degrees - of - freedom (6 - DoF) grasping postures**: Unlike traditional top - view grasping, this method outputs 6 - DoF grasping postures, making the grasping more adaptable and precise. 3. **Fuzzy reward function**: A fuzzy reward function is introduced to accelerate effective policy learning, guiding the model by evaluating the effectiveness of actions and the applicability of pushing or grasping. 4. **Simplified training and testing systems**: Two versions of the robot system are developed, one for simplified training and the other for real - world testing, ensuring a seamless transition from simulation to practical application. ### Experimental verification: The experimental results show that this framework can effectively map visual data to two - arm pushing - grasping actions in both simulated and real - world environments, and successfully complete the target - grasping tasks. Compared with other methods, this method performs better in terms of task completion rate, grasping success rate, and action efficiency, and can be transferred from the simulated environment to practical applications without additional data collection or fine - tuning. In summary, this paper significantly improves the operational performance of robots in dense and cluttered environments by introducing two - arm coordinated pushing - grasping strategies, solving the complex grasping problems that are difficult for traditional methods to handle.

Learning Dual-Arm Push and Grasp Synergy in Dense Clutter

Learning Robust Skills for Tightly Coordinated Arms in Contact-Rich Tasks

Synergistic Pushing and Grasping for Enhanced Robotic Manipulation Using Deep Reinforcement Learning

Ensemble Bootstrapped Deep Deterministic Policy Gradient For Vision-Based Robotic Grasping

Self-Supervised Learning for Joint Pushing and Grasping Policies in Highly Cluttered Environments

Position-aware pushing and grasping synergy with deep reinforcement learning in clutter

Learning Bifunctional Push-grasping Synergistic Strategy for Goal-agnostic and Goal-oriented Tasks

Harnessing the Synergy between Pushing, Grasping, and Throwing to Enhance Object Manipulation in Cluttered Scenarios

Implementation and Optimization of Grasping Learning with Dual-modal Soft Gripper.

Efficient learning of goal-oriented push-grasping synergy in clutter

DualAfford: Learning Collaborative Visual Affordance for Dual-gripper Manipulation

Deep Reinforcement Learning-Based Robotic Grasping in Clutter and Occlusion

A Dual-Arm Collaborative Framework for Dexterous Manipulation in Unstructured Environments with Contrastive Planning

Hierarchical Policies for Cluttered-Scene Grasping with Latent Plans

A Deep Learning Approach to Grasping the Invisible

Efficient push-grasping for multiple target objects in clutter environments

Deep Reinforcement Learning for Robotic Pushing and Picking in Cluttered Environment

Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representations

Collaborative Pushing and Grasping of Tightly Stacked Objects via Deep Reinforcement Learning

Reinforcement Learning Based Pushing and Grasping Objects from Ungraspable Poses

Learning Goal-Oriented Non-Prehensile Pushing in Cluttered Scenes