Performing forceful robot manipulation tasks

Melisa Yashinski

DOI: https://doi.org/10.1126/scirobotics.ado8051

IF: 25

2024-02-29

Science Robotics

Abstract:A planning framework enables a robot to perform forceful manipulation tasks, such as opening a push-twist medicine bottle.

robotics

What problem does this paper attempt to address?

Dexterous Manoeuvre Through Touch in a Cluttered Scene

Wenyu Liang,Qinyuan Ren,Xiaoqiao Chen,Junli Gao,Yan Wu

DOI: https://doi.org/10.1109/icra48506.2021.9562061

2021-01-01

Abstract:Manipulation in a densely cluttered environment creates complex challenges in perception to close the control loop, many of which are due to the sophisticated physical interaction between the environment and the manipulator. Drawing from biological sensory-motor control, to handle the task in such a scenario, tactile sensing can be used to provide an additional dimension of the rich contact information from the interaction for decision making and action selection to manoeuvre towards a target. In this paper, a new tactile-based motion planning and control framework based on bioinspiration is proposed and developed for a robot manipulator to manoeuvre in a cluttered environment. An iterative two-stage machine learning approach is used in this framework: an autoencoder is used to extract important cues from tactile sensory readings while a reinforcement learning technique is used to generate optimal motion sequence to efficiently reach the given target. The framework is implemented on a KUKA LBR iiwa robot mounted with a SynTouch BioTac tactile sensor and tested with real-life experiments. The results show that the system is able to move the end-effector through the cluttered environment to reach the target effectively.
Robust Planning for Multi-stage Forceful Manipulation

Rachel Holladay,Tomás Lozano-Pérez,Alberto Rodriguez

2023-11-21

Abstract:Multi-step forceful manipulation tasks, such as opening a push-and-twist childproof bottle, require a robot to make various planning choices that are substantially impacted by the requirement to exert force during the task. The robot must reason over discrete and continuous choices relating to the sequence of actions, such as whether to pick up an object, and the parameters of each of those actions, such how to grasp the object. To enable planning and executing forceful manipulation, we augment an existing task and motion planner with constraints that explicitly consider torque and frictional limits, captured through the proposed forceful kinematic chain constraint. In three domains, opening a childproof bottle, twisting a nut and cutting a vegetable, we demonstrate how the system selects from among a combinatorial set of strategies.We also show how cost-sensitive planning can be used to find strategies and parameters that are robust to uncertainty in the physical parameters.

Robotics
Beyond End-Effector: Utilizing High-Resolution Tactile Signals for Physical Human–Robot Interaction

Yu Sun,Cong Xiao,Lipeng Chen,Lu Chen,Haojian Lu,Yue Wang,Wang-Wei Lee,Yu Zheng,Zhengyou Zhang,Rong Xiong

DOI: https://doi.org/10.1109/tie.2024.3458180

IF: 7.7

2024-01-01

IEEE Transactions on Industrial Electronics

Abstract:This article proposes a control framework for robots to apply their entire body as potential effectors in physical human-robot interaction (pHRI) tasks. This framework is implemented based on high-resolution electronic skin that covers the entire body of the robot. During pHRI, robots must respond appropriately to human intentions, necessitating advanced sensing capabilities to interpret tactile information effectively. However, both discerning human intention from such large-scale tactile data and accommodating interaction across the entire body's surface present challenges. In this article, we propose a method to convert the large-area contact on a link into a contact center and estimate the corresponding wrench. Furthermore, we assign soft priorities and desired trajectories to each contact point and solve for the optimal joint velocities through quadratic programming (QP). By enabling a dual-arm mobile manipulator to dance the waltz with a human, our control framework has been validated for its effectiveness in handling multiple large-area contacts and time-varying pHRI tasks.
Feeling the Force: Integrating Force and Pose for Fluent Discovery Through Imitation Learning to Open Medicine Bottles.

Mark Edmonds,Feng Gao,Xu Xie,Hangxin Liu,Siyuan Qi,Yixin Zhu,Brandon Rothrock,Song-Chun Zhu

DOI: https://doi.org/10.1109/iros.2017.8206196

2017-01-01

Abstract:Learning complex robot manipulation policies for real-world objects is challenging, often requiring significant tuning within controlled environments. In this paper, we learn a manipulation model to execute tasks with multiple stages and variable structure, which typically are not suitable for most robot manipulation approaches. The model is learned from human demonstration using a tactile glove that measures both hand pose and contact forces. The tactile glove enables observation of visually latent changes in the scene, specifically the forces imposed to unlock the child-safety mechanisms of medicine bottles. From these observations, we learn an action planner through both a top-down stochastic grammar model (And-Or graph) to represent the compositional nature of the task sequence and a bottom-up discriminative model from the observed poses and forces. These two terms are combined during planning to select the next optimal action. We present a method for transferring this human-specific knowledge onto a robot platform and demonstrate that the robot can perform successful manipulations of unseen objects with similar task structure.
Manipulation planning under changing external forces

Lipeng Chen,Luis F. C. Figueredo,Mehmet R. Dogar

DOI: https://doi.org/10.1007/s10514-020-09930-z

IF: 3.255

2020-07-09

Autonomous Robots

Abstract:This paper presents a planner that enables robots to manipulate objects under changing external forces. Particularly, we focus on the scenario where a human applies a sequence of forceful operations, e.g. cutting and drilling, on an object that is held by a robot. The planner produces an efficient manipulation plan by choosing stable grasps on the object, by intelligently deciding when the robot should change its grasp on the object as the external forces change, and by choosing subsequent grasps such that they minimize the number of regrasps required in the long-term. Furthermore, as it switches from one grasp to the other, the planner solves the bimanual regrasping in the air by using an alternating sequence of bimanual and unimanual grasps. We also present a conic formulation to address force uncertainties inherent in human-applied external forces, using which the planner can robustly assess the stability of a grasp configuration without sacrificing planning efficiency. We provide a planner implementation on a dual-arm robot and present a variety of simulated and real human-robot experiments to show the performance of our planner.
Rearrangement-Based Manipulation via Kinodynamic Planning and Dynamic Planning Horizons

Kejia Ren,Lydia E. Kavraki,Kaiyu Hang

DOI: https://doi.org/10.48550/arXiv.2208.02312

2022-08-04

Abstract:Robot manipulation in cluttered environments often requires complex and sequential rearrangement of multiple objects in order to achieve the desired reconfiguration of the target objects. Due to the sophisticated physical interactions involved in such scenarios, rearrangement-based manipulation is still limited to a small range of tasks and is especially vulnerable to physical uncertainties and perception noise. This paper presents a planning framework that leverages the efficiency of sampling-based planning approaches, and closes the manipulation loop by dynamically controlling the planning horizon. Our approach interleaves planning and execution to progressively approach the manipulation goal while correcting any errors or path deviations along the process. Meanwhile, our framework allows the definition of manipulation goals without requiring explicit goal configurations, enabling the robot to flexibly interact with all objects to facilitate the manipulation of the target ones. With extensive experiments both in simulation and on a real robot, we evaluate our framework on three manipulation tasks in cluttered environments: grasping, relocating, and sorting. In comparison with two baseline approaches, we show that our framework can significantly improve planning efficiency, robustness against physical uncertainties, and task success rate under limited time budgets.

Robotics
Jacta: A Versatile Planner for Learning Dexterous and Whole-body Manipulation

Jan Brüdigam,Ali-Adeeb Abbas,Maks Sorokin,Kuan Fang,Brandon Hung,Maya Guru,Stefan Sosnowski,Jiuguang Wang,Sandra Hirche,Simon Le Cleac'h

2024-08-02

Abstract:Robotic manipulation is challenging due to discontinuous dynamics, as well as high-dimensional state and action spaces. Data-driven approaches that succeed in manipulation tasks require large amounts of data and expert demonstrations, typically from humans. Existing manipulation planners are restricted to specific systems and often depend on specialized algorithms for using demonstration. Therefore, we introduce a flexible motion planner tailored to dexterous and whole-body manipulation tasks. Our planner creates readily usable demonstrations for reinforcement learning algorithms, eliminating the need for additional training pipeline complexities. With this approach, we can efficiently learn policies for complex manipulation tasks, where traditional reinforcement learning alone only makes little progress. Furthermore, we demonstrate that learned policies are transferable to real robotic systems for solving complex dexterous manipulation tasks.

Robotics
Collision-inclusive Manipulation Planning for Occluded Object Grasping via Compliant Robot Motions

Kejia Ren,Gaotian Wang,Andrew S. Morgan,Kaiyu Hang

2024-12-10

Abstract:Traditional robotic manipulation mostly focuses on collision-free tasks. In practice, however, many manipulation tasks (e.g., occluded object grasping) require the robot to intentionally collide with the environment to reach a desired task configuration. By enabling compliant robot motions, collisions between the robot and the environment are allowed and can thus be exploited, but more physical uncertainties are introduced. To address collision-rich problems such as occluded object grasping while handling the involved uncertainties, we propose a collision-inclusive planning framework that can transition the robot to a desired task configuration via roughly modeled collisions absorbed by Cartesian impedance control. By strategically exploiting the environmental constraints and exploring inside a manipulation funnel formed by task repetitions, our framework can effectively reduce physical and perception uncertainties. With real-world evaluations on both single-arm and dual-arm setups, we show that our framework is able to efficiently address various realistic occluded grasping problems where a feasible grasp does not initially exist.

Robotics
Enhancing Dexterity in Robotic Manipulation via Hierarchical Contact Exploration

Xianyi Cheng,Sarvesh Patil,Zeynep Temel,Oliver Kroemer,Matthew T. Mason

2023-11-09

Abstract:Planning robot dexterity is challenging due to the non-smoothness introduced by contacts, intricate fine motions, and ever-changing scenarios. We present a hierarchical planning framework for dexterous robotic manipulation (HiDex). This framework explores in-hand and extrinsic dexterity by leveraging contacts. It generates rigid-body motions and complex contact sequences. Our framework is based on Monte-Carlo Tree Search and has three levels: 1) planning object motions and environment contact modes; 2) planning robot contacts; 3) path evaluation and control optimization. This framework offers two main advantages. First, it allows efficient global reasoning over high-dimensional complex space created by contacts. It solves a diverse set of manipulation tasks that require dexterity, both intrinsic (using the fingers) and extrinsic (also using the environment), mostly in seconds. Second, our framework allows the incorporation of expert knowledge and customizable setups in task mechanics and models. It requires minor modifications to accommodate different scenarios and robots. Hence, it provides a flexible and generalizable solution for various manipulation tasks. As examples, we analyze the results on 7 hand configurations and 15 scenarios. We demonstrate 8 tasks on two robot platforms.

Robotics
Planning for quasi-static manipulation tasks via an intrinsic haptic metric

Lin Yang,Sri Harsha Turlapati,Chen Lv,Domenico Campolo

2024-11-07

Abstract:Contact-rich manipulation often requires strategic interactions with objects, such as pushing to accomplish specific tasks. We propose a novel scenario where a robot inserts a book into a crowded shelf by pushing aside neighboring books to create space before slotting the new book into place. Classical planning algorithms fail in this context due to limited space and their tendency to avoid contact. Additionally, they do not handle indirectly manipulable objects or consider force interactions. Our key contributions are: i) re-framing quasi-static manipulation as a planning problem on an implicit manifold derived from equilibrium conditions; ii) utilizing an intrinsic haptic metric instead of ad-hoc cost functions; and iii) proposing an adaptive algorithm that simultaneously updates robot states, object positions, contact points, and haptic distances. We evaluate our method on such crowded bookshelf insertion task but it is a general formulation to rigid bodies manipulation tasks. We propose proxies to capture contact point and force, with superellipse to represent objects. This simplified model guarantee the differentiablity. Our framework autonomously discovers strategic wedging-in policies while our simplified contact model achieves behavior similar to real world scenarios. We also vary the stiffness and initial positions to analysis our framework comprehensively. The video can be found at <a class="link-external link-https" href="https://youtu.be/eab8umZ3AQ0" rel="external noopener nofollow">this https URL</a>.

Robotics
Task-Driven Manipulation with Reconfigurable Parallel Robots

Daniel Morton,Mark Cutkosky,Marco Pavone

2024-03-16

Abstract:ReachBot, a proposed robotic platform, employs extendable booms as limbs for mobility in challenging environments, such as martian caves. When attached to the environment, ReachBot acts as a parallel robot, with reconfiguration driven by the ability to detach and re-place the booms. This ability enables manipulation-focused scientific objectives: for instance, through operating tools, or handling and transporting samples. To achieve these capabilities, we develop a two-part solution, optimizing for robustness against task uncertainty and stochastic failure modes. First, we present a mixed-integer stance planner to determine the positioning of ReachBot's booms to maximize the task wrench space about the nominal point(s). Second, we present a convex tension planner to determine boom tensions for the desired task wrenches, accounting for the probabilistic nature of microspine grasping. We demonstrate improvements in key robustness metrics from the field of dexterous manipulation, and show a large increase in the volume of the manipulation workspace. Finally, we employ Monte-Carlo simulation to validate the robustness of these methods, demonstrating good performance across a range of randomized tasks and environments, and generalization to cable-driven morphologies. We make our code available at our project webpage,

Robotics
Improving robot manipulation with data-driven object-centric models of everyday forces

Advait Jain,Charles C. Kemp

DOI: https://doi.org/10.1007/s10514-013-9344-1

IF: 3.255

2013-06-19

Autonomous Robots

Abstract:Abstract Based on a lifetime of experience, people anticipate the forces associated with performing a manipulation task. In contrast, most robots lack common sense about the forces involved in everyday manipulation tasks. In this paper, we present data-driven methods to inform robots about the forces that they are likely to encounter when performing specific tasks. In the context of door opening, we demonstrate that data-driven object-centric models can be used to haptically recognize specific doors, haptically recognize classes of door (e.g., refrigerator vs. kitchen cabinet), and haptically detect anomalous forces while opening a door, even when opening a specific door for the first time. We also demonstrate that two distinct robots can use forces captured from people opening doors to better detect anomalous forces. These results illustrate the potential for robots to use shared databases of forces to better manipulate the world and attain common sense about everyday forces.

robotics,computer science, artificial intelligence
Preparatory Manipulation Planning using Automatically Determined Single and Dual Arms

Weiwei Wan,Kensuke Harada,Fumio Kanehiro

DOI: https://doi.org/10.48550/arXiv.1812.03274

2018-12-08

Abstract:This paper presents a manipulation planning algorithm for robots to reorient objects. It automatically finds a sequence of robot motion that manipulates and prepares an object for specific tasks. Examples of the preparatory manipulation planning problems include reorienting an electric drill to cut holes, reorienting workpieces for assembly, and reorienting cargo for packing, etc. The proposed algorithm could plan single and dual arm manipulation sequences to solve the problems. The mechanism under the planner is a regrasp graph which encodes grasp configurations and object poses. The algorithms search the graph to find a sequence of robot motion to reorient objects. The planner is able to plan both single and dual arm manipulation. It could also automatically determine whether to use a single arm, dual arms, or their combinations to finish given tasks. The planner is examined by various humanoid robots like Nextage, HRP2Kai, HRP5P, etc., using both simulation and real-world experiments.

Robotics
Integrated Planning and Control of Robotic Surgical Instruments for Tasks Autonomy

Fangxun Zhong,Yun-Hui Liu

2023-05-13

Abstract:Agile maneuvers are essential for robot-enabled complex tasks such as surgical procedures. Prior explorations on surgery autonomy are limited to feasibility study of completing a single task without systematically addressing generic manipulation safety across different tasks. We present an integrated planning and control framework for 6-DoF robotic instruments for pipeline automation of surgical tasks.We leverage the geometry of a robotic instrument and propose the nodal state space (NSS) to represent the robot state in SE(3) space. Each elementary robot motion could be encoded by regulation of the state parameters via a dynamical system. This theoretically ensures that every in-process trajectory is globally feasible and stably reached to an admissible target, and the controller is of closed-form without computing 6-DoF inverse kinematics. Then, to plan the motion steps reliably, we propose an interactive (instant) goal state of the robot that transforms manipulation planning through desired path constraints into a goal-varying manipulation (GVM) problem. We detail how GVM could adaptively and smoothly plan the procedure (could proceed or rewind the process as needed) based on on-the-fly situations under dynamic or disturbed environment. Finally, we extend the above policy to characterize complete pipelines of various surgical tasks. Simulations show that our framework could smoothly solve twisted maneuvers while avoiding collisions. Physical experiments using the da Vinci Research Kit (dVRK) validates the capability of automating individual tasks including tissue debridement, dissection, and wound suturing. The results confirm good task-level consistency and reliability compared to state-of-the-art automation algorithms.

Robotics,Systems and Control
Robot Manipulation Strategy for Explosives Disposal in Cluttered Scenes

Run Guo,Yazhou Yuan,Fuchun Sun,Shuhuan Wen,Yang Chen,Linxiang Li

2023-01-01

Abstract:Compared to human operator, robotic arm possesses the advantages of high accuracy and controllability in handling and manipulating hazardous materials. Due to the complexity and variability of application scenarios, such as the objects are stacked, it is a great challenge to accurately select target objects and manipulation mode in the fields of security and demolition. To address these problems, robotic arm systems require capabilities in target object detection, object relationship reasoning, and manipulation strategy selection. In this paper, we propose a framework with the abilities of reasoning about the unknown relationships between target objects and interfering objects, predicting manipulation sequences, and adopting appropriate manipulation strategies for hazardous object detection and manipulation processing. We conduct a series of experiments in indoor single-target, multi-target, and outdoor scenarios to complete the assigned tasks with improved manipulation sequences. These results serve as evidence of the effectiveness of the proposed framework.
A Novel Planning Framework for Complex Flipping Manipulation of Multiple Mobile Manipulators

Wenhang Liu,Meng Ren,Kun Song,Michael Yu Wang,Zhenhua Xiong

2024-10-25

Abstract:During complex object manipulation, manipulator systems often face the configuration disconnectivity problem due to closed-chain constraints. Although regrasping can be adopted to get a piecewise connected manipulation, it is a challenging problem to determine whether there is a planning result without regrasping. To address this problem, a novel planning framework is proposed for multiple mobile manipulator systems. Coordinated platform motions and regrasping motions are proposed to enhance configuration connectivity. Given the object trajectory and the grasping pose set, the planning framework includes three steps. First, inverse kinematics for each mobile manipulator is verified along the given trajectory based on different grasping poses. Coverable trajectory segments are determined for each robot for a specific grasping pose. Second, the trajectory choice problem is formulated into a set cover problem, by which we can quickly determine whether the manipulation can be completed without regrasping or with the minimal regrasping number. Finally, the motions of each mobile manipulator are planned with the assigned trajectory segments using existing methods. Both simulations and experimental results show the performance of the planner in complex flipping manipulation. Additionally, the proposed planner can greatly extend the adaptability of multiple mobile manipulator systems in complex manipulation tasks.

Robotics
Hybrid Learning- and Model-Based Planning and Control of In-Hand Manipulation

Rana Soltani Zarrin,Katsu Yamane,Rianna Jitosho

DOI: https://doi.org/10.48550/arXiv.2209.10040

2022-09-21

Abstract:This paper presents a hierarchical framework for planning and control of in-hand manipulation of a rigid object involving grasp changes using fully-actuated multifingered robotic hands. While the framework can be applied to the general dexterous manipulation, we focus on a more complex definition of in-hand manipulation, where at the goal pose the hand has to reach a grasp suitable for using the object as a tool. The high level planner determines the object trajectory as well as the grasp changes, i.e. adding, removing, or sliding fingers, to be executed by the low-level controller. While the grasp sequence is planned online by a learning-based policy to adapt to variations, the trajectory planner and the low-level controller for object tracking and contact force control are exclusively model-based to robustly realize the plan. By infusing the knowledge about the physics of the problem and the low-level controller into the grasp planner, it learns to successfully generate grasps similar to those generated by model-based optimization approaches, obviating the high computation cost of online running of such methods to account for variations. By performing experiments in physics simulation for realistic tool use scenarios, we show the success of our method on different tool-use tasks and dexterous hand models. Additionally, we show that this hybrid method offers more robustness to trajectory and task variations compared to a model-based method.

Robotics
Research on the opening method of robotic arm based on force feedback reinforcement learning

Ziyang Zhou,Liming Wang,Yang Wang

DOI: https://doi.org/10.1063/5.0167500

IF: 1.697

2024-01-01

AIP Advances

Abstract:In practical applications involving robotic arms, particularly in tasks such as manipulating door handles, improper strategies often lead to excessive contact forces. Such forces not only jeopardize the integrity of the robotic arm’s joints but also pose a risk of damaging the door handle. This paper delves into a meticulous study aimed at refining the opening techniques employed by manipulators, enhancing their adaptability across various environments. A novel method is introduced, amalgamating force information feedback with the deep deterministic policy gradient algorithm, fostering a more nuanced approach in trajectory planning. This innovative strategy is meticulously evaluated through simulations and physical experiments, proving instrumental in guiding the robotic arm toward the successful completion of the door-opening task. The findings from the experiments underscore the algorithm’s prowess in cultivating a compliant door-opening strategy, harmonizing with the force applied at the manipulator’s end. A comparative analysis with conventional methods reveals a notable reduction in the end force of the manipulator, facilitating a more efficient and secure execution of door-opening operations.

materials science, multidisciplinary,physics, applied,nanoscience & nanotechnology

Performing forceful robot manipulation tasks

Dexterous Manoeuvre Through Touch in a Cluttered Scene

Robust Planning for Multi-stage Forceful Manipulation

Beyond End-Effector: Utilizing High-Resolution Tactile Signals for Physical Human–Robot Interaction

Feeling the Force: Integrating Force and Pose for Fluent Discovery Through Imitation Learning to Open Medicine Bottles.

Manipulation planning under changing external forces

Rearrangement-Based Manipulation via Kinodynamic Planning and Dynamic Planning Horizons

Jacta: A Versatile Planner for Learning Dexterous and Whole-body Manipulation

Collision-inclusive Manipulation Planning for Occluded Object Grasping via Compliant Robot Motions

Enhancing Dexterity in Robotic Manipulation via Hierarchical Contact Exploration

Planning for quasi-static manipulation tasks via an intrinsic haptic metric

Task-Driven Manipulation with Reconfigurable Parallel Robots

Improving robot manipulation with data-driven object-centric models of everyday forces

Preparatory Manipulation Planning using Automatically Determined Single and Dual Arms

Integrated Planning and Control of Robotic Surgical Instruments for Tasks Autonomy

Robot Manipulation Strategy for Explosives Disposal in Cluttered Scenes

A Novel Planning Framework for Complex Flipping Manipulation of Multiple Mobile Manipulators

Hybrid Learning- and Model-Based Planning and Control of In-Hand Manipulation

Research on the opening method of robotic arm based on force feedback reinforcement learning