Abstract:The precise control of soft and continuum robots requires knowledge of their shape. The shape of these robots has, in contrast to classical rigid robots, infinite degrees of freedom. To partially reconstruct the shape, proprioceptive techniques use built-in sensors resulting in inaccurate results and increased fabrication complexity. Exteroceptive methods so far rely on placing reflective markers on all tracked components and triangulating their position using multiple motion-tracking cameras. Tracking systems are expensive and infeasible for deformable robots interacting with the environment due to marker occlusion and damage. Here, we present a regression approach for 3D shape estimation using a convolutional neural network. The proposed approach takes advantage of data-driven supervised learning and is capable of real-time marker-less shape estimation during inference. Two images of a robotic system are taken simultaneously at 25 Hz from two different perspectives, and are fed to the network, which returns for each pair the parameterized shape. The proposed approach outperforms marker-less state-of-the-art methods by a maximum of 4.4% in estimation accuracy while at the same time being more robust and requiring no prior knowledge of the shape. The approach can be easily implemented due to only requiring two color cameras without depth and not needing an explicit calibration of the extrinsic parameters. Evaluations on two types of soft robotic arms and a soft robotic fish demonstrate our method's accuracy and versatility on highly deformable systems in real-time. The robust performance of the approach against different scene modifications (camera alignment and brightness) suggests its generalizability to a wider range of experimental setups, which will benefit downstream tasks such as robotic grasping and manipulation.

A Novel Shape-based Robotic Sorting Approach Based on Computer Vision

Vision-Based Categorical Object Pose Estimation and Manipulation.

A Novel Robotic Grasp Detection Framework Using Low-Cost RGB-D Camera for Industrial Bin Picking

Vision-Based Robotic Object Grasping—A Deep Reinforcement Learning Approach

A Robotic Semantic Grasping Method for Pick-and-place Tasks

A Deep Learning-Based Autonomous RobotManipulator for Sorting Application

An Intelligent Robot Sorting System by Deep Learning on RGB-D Image

Efficient 3D Object Recognition for Unadjusted Bin Picking Automation

Visual Sorting of Express Parcels Based on Multi-Task Deep Learning

6D Pose Estimation with Combined Deep Learning and 3D Vision Techniques for a Fast and Accurate Object Grasping

Robotic Continuous Grasping System by Shape Transformer-Guided Multiobject Category-Level 6-D Pose Estimation

Robotic Continuous Grasping System by Shape Transformer-Guided Multi-Object Category-Level 6D Pose Estimation

Object Detection and Pose Estimation from RGB and Depth Data for Real-time, Adaptive Robotic Grasping

ViSE: Vision-Based 3D Online Shape Estimation of Continuously Deformable Robots

A Novel Deep Learning-Based Pose Estimation Method for Robotic Grasping of Axisymmetric Bodies in Industrial Stacked Scenarios

A Novel Vision-Based Multi-Task Robotic Grasp Detection Method for Multi-Object Scenes

Grasp Detection for Assembly Robots Using High-fidelity Synthetic Data

6Dof Pose Estimation for Intricately-Shaped Object with Prior Knowledge for Robotic Picking

Real-time deep learning–based image processing for pose estimation and object localization in autonomous robot applications

Robotic Grasping With Multi-View Image Acquisition and Model-Based Pose Estimation

Design and development of robot arm system for classification and sorting using machine vision