Model predictive manipulation of compliant objects with multi-objective optimizer and adversarial network for occlusion compensation
Jiaming Qi,Peng Zhou,Guangtao Ran,Han Gao,Pengyu Wang,Dongyu Li,Yufeng Gao,David Navarro-Alarcon
DOI: https://doi.org/10.1016/j.isatra.2024.05.015
IF: 7.3
2024-05-16
ISA Transactions
Abstract:Highlights • A parametric shape descriptor to efficiently characterize 3D deformations based on online curve/surface fitting. • A robust shape prediction network based on adversarial neural networks to compensate visual occlusions. • An optimization-based estimator to approximate the deformation Jacobian matrix and satisfy various performance constraints. • An MPC-based controller to guide the shaping motions while simultaneously solving saturation, workspace, and obstacle constraints. Background: The manipulation of compliant objects by robotic systems remains a challenging task, largely due to their variable shapes and the complex, high-dimensional nature of their interaction dynamics. Traditional robotic manipulation strategies struggle with the accurate modeling and control necessary to handle such materials, especially in the presence of visual occlusions that frequently occur in dynamic environments. Meanwhile, for most unstructured environments, robots are required to have autonomous interactions with their surroundings. Methods: To solve the shape manipulation of compliant objects in an unstructured environment, we begin by exploring the regression-based algorithm of representing the high-dimensional configuration space of deformable objects in a compressed form that enables efficient and effective manipulation. Simultaneously, we address the issue of visual occlusions by proposing the integration of an adversarial network, enabling guiding the shaping task even with partial observations of the object. Afterwards, we propose a receding-time estimator to coordinate the robot action with the computed shape features while satisfying various performance criteria. Finally, model predictive controller is utilized to compute the robot's shaping motions subject to safety constraints. Detailed experiments are presented to evaluate the proposed manipulation framework. Significant findings: Our MPC framework utilizes the compressed representation and occlusion-compensated information to predict the object's behavior, while the multi-objective optimizer ensures that the resulting control actions meet multiple performance criteria. Through rigorous experimental validation, our approach demonstrates superior manipulation capabilities in scenarios with visual obstructions, outperforming existing methods in terms of precision and operational reliability. The findings highlight the potential of our integrated approach to significantly enhance the manipulation of compliant objects in real-world robotic applications.
automation & control systems,instruments & instrumentation,engineering, multidisciplinary