A Survey on Robotic Manipulation of Deformable Objects: Recent Advances, Open Challenges and New Frontiers

Feida Gu,Yanmin Zhou,Zhipeng Wang,Shuo Jiang,Bin He

2023-12-16

Abstract:Deformable object manipulation (DOM) for robots has a wide range of applications in various fields such as industrial, service and health care sectors. However, compared to manipulation of rigid objects, DOM poses significant challenges for robotic perception, modeling and manipulation, due to the infinite dimensionality of the state space of deformable objects (DOs) and the complexity of their dynamics. The development of computer graphics and machine learning has enabled novel techniques for DOM. These techniques, based on data-driven paradigms, can address some of the challenges that analytical approaches of DOM face. However, some existing reviews do not include all aspects of DOM, and some previous reviews do not summarize data-driven approaches adequately. In this article, we survey more than 150 relevant studies (data-driven approaches mainly) and summarize recent advances, open challenges, and new frontiers for aspects of perception, modeling and manipulation for DOs. Particularly, we summarize initial progress made by Large Language Models (LLMs) in robotic manipulation, and indicates some valuable directions for further research. We believe that integrating data-driven approaches and analytical approaches can provide viable solutions to open challenges of DOM.

Robotics

What problem does this paper attempt to address?

The paper primarily explores the research progress, challenges, and future research directions in the field of Deformable Object Manipulation (DOM) by robots. Specifically, the paper aims to address the following key issues:

1. **Review of Existing Research**: The authors reviewed over 150 related papers, with a particular focus on data-driven methods, and summarized the latest advancements in the perception, modeling, and manipulation of deformable objects.

2. **Identifying Challenges and Shortcomings**: The paper points out the shortcomings in previous reviews, such as the failure to comprehensively cover all aspects of DOM or to adequately summarize data-driven methods. Additionally, it emphasizes the importance of combining analytical methods with data-driven approaches.

3. **Proposing Future Research Directions**: The paper specifically discusses the potential applications of large language models (LLMs) in the field of robotic manipulation and highlights areas worth further exploration.

4. **Addressing Key Technical Issues in DOM**: The paper delves into the specific challenges and solutions in three key technical areas of DOM—perception, modeling, and manipulation. For example, in the perception section, the paper discusses visual perception and tactile perception technologies; in the modeling section, it introduces analytical methods (such as mass-spring-damper systems, position-based dynamics, etc.) and data-driven methods (such as Jacobian-based models, graph neural networks, etc.); in the manipulation section, it covers traditional planning and control methods as well as learning-based methods.

Through the above work, the paper aims to provide researchers and practitioners with a comprehensive reference guide on robotic manipulation of deformable objects, to promote further development and technological advancement in this field.