Abstract:The manipulation of deformable objects, especially deformable linear objects (DLOs), represents an open challenge in robotics. The keys to solving the problem are accurately recognizing the topological model describing the entwined state of DLO(s) and a manipulation strategy based on it. The situation becomes more challenging in practical applications since the DLO(s) may be placed under complex and unencountered environments, making distinguishing between the targets and the background difficult. In addition, with the information obtained from a sensor system with a limited field of sense, a robot has to treat the encountered DLO(s) as multiple ones of an unknown quantity. This paper proposes a solution based on deep learning techniques for these complicated scenarios. The approach can derive a topological model describing the entangled structure of single or multiple DLOs. Based on the model, we proposed a strategy for untangling the DLO(s), considering both the possible self-tangling in one DLO and the tangling between multiple DLOs. The strategy ensures that the entangled DLO(s) can be arranged to be the neatest state given a limited field of view. The feasibility and effectiveness of the proposed solution were verified by untangling experiments utilizing a dual-arm robot system. Even if the exact quantity of the DLO(s) is unknown, the robots can still untangle the DLO(s). Moreover, the proposed approach performed robustness to unfamiliar background textures, which is preferable in practical applications. Dataset used in this paper can be found at https://github.com/lancexz/dlos-dataset. Note to Practitioners—This article proposes an automatic solution for untangling an unknown quantity of DLO(s) using robots. Issues concerned with complex backgrounds, an unknown quantity of DLOs, and limits of field of view, which are easily encountered in practical applications, are solved. Experiments demonstrated the feasibility and robustness of the approach in different scenarios. Potential applications for the proposed method include pipeline bin-picking tasks, rope-like object manipulation in household robots, and cable assembly in industrial areas. Moreover, the topological state recognition process can also be utilized in knot-tying studies and producing datasets. As the current system can not deal with scenes where DLO segments or crossings severely overlap, practitioners should integrate real-time tracking, deformation control method, and unqualified cases detection to avoid the appearance of such cases during manipulation. Future works should also consider the rebound caused by the elastic potential of DLOs.

FASTDLO: Fast Deformable Linear Objects Instance Segmentation

RT-DLO: Real-Time Deformable Linear Objects Instance Segmentation

Automatic Image Generation Pipeline for Instance Segmentation of Deformable Linear Objects

mBEST: Realtime Deformable Linear Object Detection Through Minimal Bending Energy Skeleton Pixel Traversals

LUDO: Low-Latency Understanding of Highly Deformable Objects using Point Cloud Occupancy Functions

SOLOv2: Dynamic and Fast Instance Segmentation

Fast Video Object Segmentation Via Dynamic Targeting Network

Learning to Estimate 3-D States of Deformable Linear Objects from Single-Frame Occluded Point Clouds

SODAR: Exploring Locally Aggregated Learning of Mask Representations for Instance Segmentation

DeVOS: Flow-Guided Deformable Transformer for Video Object Segmentation

Simultaneous Shape Tracking of Multiple Deformable Linear Objects with Global-Local Topology Preservation

SOLO: A Simple Framework for Instance Segmentation

Contrastive Lift: 3D Object Instance Segmentation by Slow-Fast Contrastive Fusion

Open-YOLO 3D: Towards Fast and Accurate Open-Vocabulary 3D Instance Segmentation

Untangling Multiple Deformable Linear Objects in Unknown Quantities With Complex Backgrounds

Deep-DM: Deep-driven deformable model for 3D image segmentation using limited data

Fast Video Object Segmentation via Dynamic YOLACT

Deep-learning Optical Flow for Measuring Velocity Fields from Experimental Data

INSTA-YOLO: Real-Time Instance Segmentation

FreeSOLO: Learning to Segment Objects without Annotations

DPPD: Deformable Polar Polygon Object Detection