Abstract:The manipulation of deformable objects, especially deformable linear objects (DLOs), represents an open challenge in robotics. The keys to solving the problem are accurately recognizing the topological model describing the entwined state of DLO(s) and a manipulation strategy based on it. The situation becomes more challenging in practical applications since the DLO(s) may be placed under complex and unencountered environments, making distinguishing between the targets and the background difficult. In addition, with the information obtained from a sensor system with a limited field of sense, a robot has to treat the encountered DLO(s) as multiple ones of an unknown quantity. This paper proposes a solution based on deep learning techniques for these complicated scenarios. The approach can derive a topological model describing the entangled structure of single or multiple DLOs. Based on the model, we proposed a strategy for untangling the DLO(s), considering both the possible self-tangling in one DLO and the tangling between multiple DLOs. The strategy ensures that the entangled DLO(s) can be arranged to be the neatest state given a limited field of view. The feasibility and effectiveness of the proposed solution were verified by untangling experiments utilizing a dual-arm robot system. Even if the exact quantity of the DLO(s) is unknown, the robots can still untangle the DLO(s). Moreover, the proposed approach performed robustness to unfamiliar background textures, which is preferable in practical applications. Dataset used in this paper can be found at https://github.com/lancexz/dlos-dataset. Note to Practitioners—This article proposes an automatic solution for untangling an unknown quantity of DLO(s) using robots. Issues concerned with complex backgrounds, an unknown quantity of DLOs, and limits of field of view, which are easily encountered in practical applications, are solved. Experiments demonstrated the feasibility and robustness of the approach in different scenarios. Potential applications for the proposed method include pipeline bin-picking tasks, rope-like object manipulation in household robots, and cable assembly in industrial areas. Moreover, the topological state recognition process can also be utilized in knot-tying studies and producing datasets. As the current system can not deal with scenes where DLO segments or crossings severely overlap, practitioners should integrate real-time tracking, deformation control method, and unqualified cases detection to avoid the appearance of such cases during manipulation. Future works should also consider the rebound caused by the elastic potential of DLOs.

Untying cable by combining 3D deep neural network with deep reinforcement learning

Learning Robot Manipulation Skills from Human Demonstration Videos Using Two-Stream 2-D/3-D Residual Networks with Self-Attention

Autonomously Untangling Long Cables

Utilizing Reinforcement Learning to Drive Redundant Constrained Cable-Driven Robots with Unknown Parameters

Untangling Dense Knots by Learning Task-Relevant Keypoints

Vision Based Cable Assembly in Constrained Environment.

Offline-Online Learning of Deformation Model for Cable Manipulation With Graph Neural Networks

Target-free recognition of cable vibration in complex backgrounds based on computer vision

Tangle- and contact-free path planning for a tethered mobile robot using deep reinforcement learning

Learning-based Cable Coupling Effect Modeling for Robotic Manipulation of Heavy Industrial Cables

Generation of realistic synthetic cable images to train deep learning segmentation models

Multi-Stage Cable Routing through Hierarchical Imitation Learning

Cable manipulation with a tactile-reactive gripper

Multistage Cable Routing Through Hierarchical Imitation Learning

SGTM 2.0: Autonomously Untangling Long Cables using Interactive Perception

Cable Routing and Assembly using Tactile-driven Motion Primitives

Adaptive Disassembly Sequence Planning for VR Maintenance Training Via Deep Reinforcement Learning

Folding Knots Using a Team of Aerial Robots

Self-Supervised Learning of Dynamic Planar Manipulation of Free-End Cables

Untangling Multiple Deformable Linear Objects in Unknown Quantities With Complex Backgrounds

Dynamic Obstacle Avoidance for Cable-Driven Parallel Robots With Mobile Bases via Sim-to-Real Reinforcement Learning