Abstract:Convolutional neural networks (CNNs) have been widely used for object recognition and grasping posture planning in intelligent robotic grasping (IRG). Compared with the traditional usage of CNNs in image recognition, IRGs require high recognition accuracy and computational efficiency. However, the existing methodologies for CNN architecture design often rely on human experience and numerous trial-and-error attempts, which make it a very challenging task to obtain an optimal CNN for IRGs. To tackle this challenge, this paper develops a new differentiable architecture search (DARTS) method considering the floating-point operations (FLOPs) of CNNs, named the DARTS-F method, which converts the discrete CNN architecture search to a gradient-based continuous optimization problem and considers both the prediction accuracy and the computational cost of the CNN during the optimization. To efficiently identify the optimal neural network, this paper adopts a bilevel optimization, which first trains the neural network weights in the inner level and then optimizes the neural network architecture by fine-tuning the operational variables in the outer level. In addition, a new digital twin (DT) of IRG is developed considering the physics of realistic robotic grasping in the DT’s virtual space, which could not only improve the IRG accuracy but also avoid the expensive training time. In the experiments, the proposed DARTS-F method could generate an optimized CNN with higher prediction accuracy and lower FLOPs than those obtained by the original DARTS method. The DT framework improves the accuracy of real robotic grasping from 61 to 71%.

Target Recognition and Optimal Grasping Based on Deep Learning

A Cascaded Deep Learning Framework for Real-time and Robust Grasp Planning

DexRepNet: Learning Dexterous Robotic Grasping Network with Geometric and Spatial Hand-Object Representations

Robot grasping system based on deep learning target detection

Target Recognition and Location Based on Deep Learning

Vision-Based Robotic Object Grasping—A Deep Reinforcement Learning Approach

Simulation and Deep Learning on Point Clouds for Robot Grasping

A learning framework for semantic reach-to-grasp tasks integrating machine learning and optimization.

Deep learning-based method for vision-guided robotic grasping of unknown objects

Towards Robotic Picking of Targets with Background Distractors using Deep Reinforcement Learning

6D Pose Estimation with Combined Deep Learning and 3D Vision Techniques for a Fast and Accurate Object Grasping

A new differentiable architecture search method for optimizing convolutional neural networks in the digital twin of intelligent robotic grasping

Deep learning for detecting robotic grasps

Deep Learning Method for Grasping Novel Objects Using Dexterous Hands

Vision-based robotic grasping using faster R-CNN–GRCNN dual-layer detection mechanism

A Robotic Grasping Algorithm Based on Simplified Image and Deep Convolutional Neural Network

Deep Vision Networks for Real-Time Robotic Grasp Detection

Robotic Grasp Detection Network Based on Improved Deformable Convolution and Spatial Feature Center Mechanism

Autonomous Robotic Manipulation: Real-Time, Deep-Learning Approach for Grasping of Unknown Objects

Object Discovery and Grasp Detection with a Shared Convolutional Neural Network

Review of Deep Learning Methods in Robotic Grasp Detection