Abstract:Highlights • A parametric shape descriptor to efficiently characterize 3D deformations based on online curve/surface fitting. • A robust shape prediction network based on adversarial neural networks to compensate visual occlusions. • An optimization-based estimator to approximate the deformation Jacobian matrix and satisfy various performance constraints. • An MPC-based controller to guide the shaping motions while simultaneously solving saturation, workspace, and obstacle constraints. Background: The manipulation of compliant objects by robotic systems remains a challenging task, largely due to their variable shapes and the complex, high-dimensional nature of their interaction dynamics. Traditional robotic manipulation strategies struggle with the accurate modeling and control necessary to handle such materials, especially in the presence of visual occlusions that frequently occur in dynamic environments. Meanwhile, for most unstructured environments, robots are required to have autonomous interactions with their surroundings. Methods: To solve the shape manipulation of compliant objects in an unstructured environment, we begin by exploring the regression-based algorithm of representing the high-dimensional configuration space of deformable objects in a compressed form that enables efficient and effective manipulation. Simultaneously, we address the issue of visual occlusions by proposing the integration of an adversarial network, enabling guiding the shaping task even with partial observations of the object. Afterwards, we propose a receding-time estimator to coordinate the robot action with the computed shape features while satisfying various performance criteria. Finally, model predictive controller is utilized to compute the robot's shaping motions subject to safety constraints. Detailed experiments are presented to evaluate the proposed manipulation framework. Significant findings: Our MPC framework utilizes the compressed representation and occlusion-compensated information to predict the object's behavior, while the multi-objective optimizer ensures that the resulting control actions meet multiple performance criteria. Through rigorous experimental validation, our approach demonstrates superior manipulation capabilities in scenarios with visual obstructions, outperforming existing methods in terms of precision and operational reliability. The findings highlight the potential of our integrated approach to significantly enhance the manipulation of compliant objects in real-world robotic applications.

Collaborative Manipulation of Deformable Objects with Predictive Obstacle Avoidance

Dynamical Obstacle Avoidance of Task- Constrained Mobile Manipulation Using Model Predictive Control

Model predictive manipulation of compliant objects with multi-objective optimizer and adversarial network for occlusion compensation

A Model Predictive Approach for Online Mobile Manipulation of Nonholonomic Objects using Learned Dynamics

Kinematic-Model-Free Predictive Control for Robotic Manipulator Target Reaching With Obstacle Avoidance

Obstacle Avoidance for Manipulator with Arbitrary Arm Shape Using Signed Distance Function

Hierarchical Adaptive Motion Planning with Nonlinear Model Predictive Control for Safety-Critical Collaborative Loco-Manipulation

Decentralized Ability-Aware Adaptive Control for Multi-robot Collaborative Manipulation

DeRi-Bot: Learning to Collaboratively Manipulate Rigid Objects via Deformable Objects

Hierarchical Adaptive Control for Collaborative Manipulation of a Rigid Object by Quadrupedal Robots

Generalizable whole-body global manipulation of deformable linear objects by dual-arm robot in 3-D constrained environments

Human-Robot Collaborative Carrying of Objects with Unknown Deformation Characteristics

Differentiable Particles for General-Purpose Deformable Object Manipulation

Real-Time Deformable-Contact-Aware Model Predictive Control for Force-Modulated Manipulation

Real-time Collision Avoidance for Robot Manipulators by Braking Trajectory Prediction

Enhancing Deformable Object Manipulation By Using Interactive Perception and Assistive Tools

On-line collision avoidance for collaborative robot manipulators by adjusting off-line generated paths: An industrial use case

Enforcing Constraints for Dynamic Obstacle Avoidance by Compliant Robots

Online Collision Avoidance for Human-Robot Collaborative Interaction Concerning Safety and Efficiency

Collaborative Multi-Robot Transportation in Obstacle-Cluttered Environments via Implicit Communication

Manipulator Motion Planning Using Flexible Obstacle Avoidance Based on Model Learning