Soft Contact Simulation and Manipulation Learning of Deformable Objects with Vision-based Tactile Sensor
Jianhua Shan,Yuhao Sun,Shixin Zhang,Fuchun Sun,Zixi Chen,Zirong Shen,Cesare Stefanini,Yiyong Yang,Shan Luo,Bin Fang
DOI: https://doi.org/10.48550/arxiv.2405.07237
2024-01-01
Abstract:Deformable object manipulation is a classical and challenging research areain robotics. Compared with rigid object manipulation, this problem is morecomplex due to the deformation properties including elastic, plastic, andelastoplastic deformation. In this paper, we describe a new deformable objectmanipulation method including soft contact simulation, manipulation learning,and sim-to-real transfer. We propose a novel approach utilizing Vision-BasedTactile Sensors (VBTSs) as the end-effector in simulation to produceobservations like relative position, squeezed area, and object contour, whichare transferable to real robots. For a more realistic contact simulation, a newsimulation environment including elastic, plastic, and elastoplasticdeformations is created. We utilize RL strategies to train agents in thesimulation, and expert demonstrations are applied for challenging tasks.Finally, we build a real experimental platform to complete the sim-to-realtransfer and achieve a 90sphere. To test the robustness of our method, we use plasticine of differenthardness and sizes to repeat the tasks including cylinder and sphere. Theexperimental results show superior performances of deformable objectmanipulation with the proposed method.