Shared control–based bimanual robot manipulation

Daniel Rakita,Bilge Mutlu,Michael Gleicher,Laura M Hiatt,Laura M. Hiatt

DOI: https://doi.org/10.1126/scirobotics.aaw0955

IF: 25

2019-05-22

Science Robotics

Abstract:A bimanual shared-control method enables novice users to control a robot in real time to complete complex manipulation tasks.

robotics

What problem does this paper attempt to address?

Collaborative Bimanual Manipulation Using Optimal Motion Adaptation and Interaction Control

Ruoshi Wen,Quentin Rouxel,Michael Mistry,Zhibin Li,Carlo Tiseo

DOI: https://doi.org/10.1109/MRA.2023.3270222

2023-07-18

Abstract:This work developed collaborative bimanual manipulation for reliable and safe human-robot collaboration, which allows remote and local human operators to work interactively for bimanual tasks. We proposed an optimal motion adaptation to retarget arbitrary commands from multiple human operators into feasible control references. The collaborative manipulation framework has three main modules: (1) contact force modulation for compliant physical interactions with objects via admittance control; (2) task-space sequential equilibrium and inverse kinematics optimization, which adapts interactive commands from multiple operators to feasible motions by satisfying the task constraints and physical limits of the robots; and (3) an interaction controller adopted from the fractal impedance control, which is robust to time delay and stable to superimpose multiple control efforts for generating desired joint torques and controlling the dual-arm robots. Extensive experiments demonstrated the capability of the collaborative bimanual framework, including (1) dual-arm teleoperation that adapts arbitrary infeasible commands that violate joint torque limits into continuous operations within safe boundaries, compared to failures without the proposed optimization; (2) robust maneuver of a stack of objects via physical interactions in presence of model inaccuracy; (3) collaborative multi-operator part assembly, and teleoperated industrial connector insertion, which validate the guaranteed stability of reliable human-robot co-manipulation.

Robotics
Human–Robot Shared Control Based on Locally Weighted Intent Prediction for a Teleoperated Hydraulic Manipulator System

Shaqi Luo,Min Cheng,Ruqi Ding,Feng Wang,Bing Xu,Bingkui Chen

DOI: https://doi.org/10.1109/tmech.2022.3157852

2022-01-01

IEEE/ASME Transactions on Mechatronics

Abstract:Human–robot shared control provides a promising opportunity to improve operational safety and efficiency for a teleoperated hydraulic manipulator system that works in unstructured environments, such as rescue response and underwater exploration. However, the poor dynamic of the heavy-duty manipulator (e.g., oscillation tendency and response delay) brings a challenging problem on intent prediction and effective assistance. To overcome this problem, a blended shared control method is proposed via locally weighted intent prediction. First, a task learning method from demonstration is presented by task-based action primitives with the Bayesian nonparametric clustering to extract subgoals of the remote arm trajectory, especially a locally weighted chattering cancellation method is proposed to reduce unintended action primitives due to velocity fluctuations. A dynamic intent prediction method is presented based on the empirical and real-time motion information by introducing an iterative transition matrix and a locally weighted dynamic angle, respectively. A prediction-based arbitration rule is finally established to blend the controllers of the human and robot seamlessly. Comparative tests in two typical scenes were carried out on a teleoperated hydraulic manipulator system. The results validate the proposed shared control method with fewer collision accidents and reduced task completion time, and subjective feelings on operation are also improved.
Brain-Actuated Control of Dual-Arm Robot Manipulation With Relative Motion

Zhijun Li,Wang Yuan,Suna Zhao,Zhenzhong Yu,Yu Kang,C. L. Philip Chen

DOI: https://doi.org/10.1109/TCDS.2017.2770168

IF: 4.546

2019-01-01

IEEE Transactions on Cognitive and Developmental Systems

Abstract:This paper describes the brain-actuated control of a dual-arm robot performing bimanual relative motion manipulation tasks, through the adoption of relative Jacobian matrix, wherein the dual-arm robot can be considered as a single manipulator and the movements of the end effectors can be calculated by the relative motion. An online brain–machine interface (BMI) system based on multichannel steady-state visual evoked potentials was developed, and it was able to perform band-pass filtering and visual stimuli classification using support vector machines. Considering the relative motion in a constrained plane, the asymmetric bimanual manipulation can be transformed into 2-D control tasks through polar coordinate transformation such that the end effectors can achieve smooth direction-and-distance motion in the arbitrary position of the operational space. Moreover, the kinematics redundancy scheme, using online neuro-dynamics optimization, was developed for joint velocity optimization subject to physical constraints. Five individuals participated in the experiments and successfully fulfilled the given manipulation task.
An Intent-based Task-aware Shared Control Framework for Intuitive Hands Free Telemanipulation

Michael Bowman,Jiucai Zhang,Xiaoli Zhang

DOI: https://doi.org/10.48550/arXiv.2003.03677

2021-06-01

Abstract:Shared control in teleoperation for providing robot assistance to accomplish object manipulation, called telemanipulation, is a new promising yet challenging problem. This has unique challenges--on top of teleoperation challenges in general--due to difficulties of physical discrepancy between human hands and robot hands as well as the fine motion constraints to constitute task success. We present an intuitive shared-control strategy where the focus is on generating robotic grasp poses which are better suited for human perception of successful teleoperated object manipulation and feeling of being in control of the robot, rather than developing objective stable grasp configurations for task success or following the human motion. The former is achieved by understanding human intent and autonomously taking over control on that inference. The latter is achieved by considering human inputs as hard motion constraints which the robot must abide. An arbitration of these two enables a trade-off for the subsequent robot motion to balance accomplishing the inferred task and motion constraints imposed by the operator. The arbitration framework adapts to the level of physical discrepancy between the human and different robot structures, enabling the assistance to indicate and appear to intuitively follow the user. To understand how users perceive good arbitration in object telemanipulation, we have conducted a user study with a hands-free telemanipulation setup to analyze the effect of factors including task predictability, perceived following, and user preference. The hands-free telemanipulation scene is chosen as the validation platform due to its more urgent need of intuitive robotics assistance for task success.

Robotics
Robust Shared Control With Stable Contact Servoing for Enhanced Object Transportation by Telerobotic Bimanual Mobile Manipulators

Tao Teng,Xi Wu,Zhihao Li,Darwin Caldwell,Fei Chen

DOI: https://doi.org/10.1002/rob.22464

IF: 6.385

2024-11-10

Journal of Field Robotics

Abstract:Bimanual mobile manipulation significantly enhances the capabilities of field robots that interact with diverse dynamic environments and expands the range of possible manipulation actions. This paper introduces a robust shared control architecture for remote manipulation of the Collaborative Dual‐arm Robot Manipulator (CURI), aimed at stable object transportation. Given that stable contact is pivotal for successful object transportation, our focus is on stabilizing the contact between the robot end‐effectors and the object using a contact servoing strategy. To maintain stable contact, it is essential that the contact wrenches consistently stay within the predefined friction cones throughout the task. We propose a novel shared control strategy to stabilize the contact using the contact parameterization model. This model formulates the contact stability manifold using a set of unconstrained reparameterized variables through a bijective mapping function, ensuring that the controller derived from these unconstrained reparameterized variables consistently maintains contact stability. The efficacy of the proposed approach is convincingly demonstrated through a series of experiments involving representative bimanual transportation tasks within a logistics context. Both theoretical analysis and experimental validations confirm that our proposed control strategy not only ensures the stable transportation of objects by CURI but also substantially enhances the overall robustness of the shared control system.

robotics
Human-robot Shared Control for Humanoid Manipulator Trajectory Planning

Yuanchao Zhu,Canjun Yang,Qianxiao Wei,Xin Wu,Wei Yang

DOI: https://doi.org/10.1108/ir-10-2019-0217

2020-01-01

Industrial Robot the international journal of robotics research and application

Abstract:Purpose This paper aims to propose an intuitive shared control strategy to control a humanoid manipulator that can fully combine the advantages of humans and machines to produce a stronger intelligent form. Design/methodology/approach The working space of an operator’s arm and that of a manipulator are matched, and a genetic algorithm that limits the position of the manipulator’s elbow joint is used to find the optimal solution. Then, the mapping of the operator’s action to that of manipulators is realized. The controls of the human and robot are integrated. First, the current action of the operator is input. Second, the target object is predicted according to the maximum entropy hypothesis. Third, the joint angle of the manipulator is interpolated based on time. Finally, the confidence and weight of the current moment are calculated. Findings The modified weight adjustment method is the optimal way to adjust the weight during the task. In terms of time and accuracy, the experimental results of single target obstacle avoidance grabbing and multi-target predictive grabbing show that the shared control mode can provide full play to the advantages of humans and robots to accomplish the target task faster and more accurately than the control merely by a human or robot on its own. Originality/value A flexible and highly anthropomorphic human–robot action mapping method is proposed, which provides operator decisions in the shared control process. The shared control between human and the robot is realized, and it enhances the rapidity and intelligence, paving a new way for a novel human–robot collaboration.
Understanding Shared Control for Assistive Robotic Arms

Kirill Kronhardt,Max Pascher,Jens Gerken

2023-03-03

Abstract:Living a self-determined life independent of human caregivers or fully autonomous robots is a crucial factor for human dignity and the preservation of self-worth for people with motor impairments. Assistive robotic solutions - particularly robotic arms - are frequently deployed in domestic care, empowering people with motor impairments in performing ADLs independently. However, while assistive robotic arms can help them perform ADLs, currently available controls are highly complex and time-consuming due to the need to control multiple DoFs at once and necessary mode-switches. This work provides an overview of shared control approaches for assistive robotic arms, which aim to improve their ease of use for people with motor impairments. We identify three main takeaways for future research: Less is More, Pick-and-Place Matters, and Communicating Intent.

Human-Computer Interaction,Robotics
Bi-Manual Manipulation and Attachment via Sim-to-Real Reinforcement Learning

Satoshi Kataoka,Seyed Kamyar Seyed Ghasemipour,Daniel Freeman,Igor Mordatch

DOI: https://doi.org/10.48550/arXiv.2203.08277

IF: 3.7

2022-03-15

Robotics

Abstract:Most successes in robotic manipulation have been restricted to single-arm robots, which limits the range of solvable tasks to pick-and-place, insertion, and objects rearrangement. In contrast, dual and multi arm robot platforms unlock a rich diversity of problems that can be tackled, such as laundry folding and executing cooking skills. However, developing controllers for multi-arm robots is complexified by a number of unique challenges, such as the need for coordinated bimanual behaviors, and collision avoidance amongst robots. Given these challenges, in this work we study how to solve bi-manual tasks using reinforcement learning (RL) trained in simulation, such that the resulting policies can be executed on real robotic platforms. Our RL approach results in significant simplifications due to using real-time (4Hz) joint-space control and directly passing unfiltered observations to neural networks policies. We also extensively discuss modifications to our simulated environment which lead to effective training of RL policies. In addition to designing control algorithms, a key challenge is how to design fair evaluation tasks for bi-manual robots that stress bimanual coordination, while removing orthogonal complicating factors such as high-level perception. In this work, we design a Connect Task, where the aim is for two robot arms to pick up and attach two blocks with magnetic connection points. We validate our approach with two xArm6 robots and 3D printed blocks with magnetic attachments, and find that our system has 100% success rate at picking up blocks, and 65% success rate at the Connect Task.
Shared-Control Teleoperation Paradigms on a Soft Growing Robot Manipulator

Fabio Stroppa,Mario Selvaggio,Nathaniel Agharese,MingLuo,Laura H. Blumenschein,Elliot W. Hawkes,Allison M. Okamura

DOI: https://doi.org/10.1007/s10846-023-01919-x

2023-06-17

Abstract:Semi-autonomous telerobotic systems allow both humans and robots to exploit their strengths, while enabling personalized execution of a task. However, for new soft robots with degrees of freedom dissimilar to those of human operators, it is unknown how the control of a task should be divided between the human and robot. This work presents a set of interaction paradigms between a human and a soft growing robot manipulator, and demonstrates them in both real and simulated scenarios. The robot can grow and retract by eversion and inversion of its tubular body, a property we exploit to implement interaction paradigms. We implemented and tested six different paradigms of human-robot interaction, beginning with full teleoperation and gradually adding automation to various aspects of the task execution. All paradigms were demonstrated by two expert and two naive operators. Results show that humans and the soft robot manipulator can split control along degrees of freedom while acting simultaneously. In the simple pick-and-place task studied in this work, performance improves as the control is gradually given to the robot, because the robot can correct certain human errors. However, human engagement and enjoyment may be maximized when the task is at least partially shared. Finally, when the human operator is assisted by haptic feedback based on soft robot position errors, we observed that the improvement in performance is highly dependent on the expertise of the human operator.

Robotics
A System for Imitation Learning of Contact-Rich Bimanual Manipulation Policies

Simon Stepputtis,Maryam Bandari,Stefan Schaal,Heni Ben Amor

DOI: https://doi.org/10.48550/arXiv.2208.00596

IF: 3.7

2022-08-01

Robotics

Abstract:In this paper, we discuss a framework for teaching bimanual manipulation tasks by imitation. To this end, we present a system and algorithms for learning compliant and contact-rich robot behavior from human demonstrations. The presented system combines insights from admittance control and machine learning to extract control policies that can (a) recover from and adapt to a variety of disturbances in time and space, while also (b) effectively leveraging physical contact with the environment. We demonstrate the effectiveness of our approach using a real-world insertion task involving multiple simultaneous contacts between a manipulated object and insertion pegs. We also investigate efficient means of collecting training data for such bimanual settings. To this end, we conduct a human-subject study and analyze the effort and mental demand as reported by the users. Our experiments show that, while harder to provide, the additional force/torque information available in teleoperated demonstrations is crucial for phase estimation and task success. Ultimately, force/torque data substantially improves manipulation robustness, resulting in a 90% success rate in a multipoint insertion task. Code and videos can be found at https://bimanualmanipulation.com/
Shared Control of Bimanual Robotic Limbs With a Brain-Machine Interface for Self-Feeding

David A Handelman,Luke E Osborn,Tessy M Thomas,Andrew R Badger,Margaret Thompson,Robert W Nickl,Manuel A Anaya,Jared M Wormley,Gabriela L Cantarero,David McMullen,Nathan E Crone,Brock Wester,Pablo A Celnik,Matthew S Fifer,Francesco V Tenore

DOI: https://doi.org/10.3389/fnbot.2022.918001

2022-06-28

Abstract:Advances in intelligent robotic systems and brain-machine interfaces (BMI) have helped restore functionality and independence to individuals living with sensorimotor deficits; however, tasks requiring bimanual coordination and fine manipulation continue to remain unsolved given the technical complexity of controlling multiple degrees of freedom (DOF) across multiple limbs in a coordinated way through a user input. To address this challenge, we implemented a collaborative shared control strategy to manipulate and coordinate two Modular Prosthetic Limbs (MPL) for performing a bimanual self-feeding task. A human participant with microelectrode arrays in sensorimotor brain regions provided commands to both MPLs to perform the self-feeding task, which included bimanual cutting. Motor commands were decoded from bilateral neural signals to control up to two DOFs on each MPL at a time. The shared control strategy enabled the participant to map his four-DOF control inputs, two per hand, to as many as 12 DOFs for specifying robot end effector position and orientation. Using neurally-driven shared control, the participant successfully and simultaneously controlled movements of both robotic limbs to cut and eat food in a complex bimanual self-feeding task. This demonstration of bimanual robotic system control via a BMI in collaboration with intelligent robot behavior has major implications for restoring complex movement behaviors for those living with sensorimotor deficits.
PerAct2: Benchmarking and Learning for Robotic Bimanual Manipulation Tasks

Markus Grotz,Mohit Shridhar,Tamim Asfour,Dieter Fox

2024-08-01

Abstract:Bimanual manipulation is challenging due to precise spatial and temporal coordination required between two arms. While there exist several real-world bimanual systems, there is a lack of simulated benchmarks with a large task diversity for systematically studying bimanual capabilities across a wide range of tabletop tasks. This paper addresses the gap by extending RLBench to bimanual manipulation. We open-source our code and benchmark comprising 13 new tasks with 23 unique task variations, each requiring a high degree of coordination and adaptability. To kickstart the benchmark, we extended several state-of-the art methods to bimanual manipulation and also present a language-conditioned behavioral cloning agent -- PerAct2, which enables the learning and execution of bimanual 6-DoF manipulation tasks. Our novel network architecture efficiently integrates language processing with action prediction, allowing robots to understand and perform complex bimanual tasks in response to user-specified goals. Project website with code is available at: <a class="link-external link-http" href="http://bimanual.github.io" rel="external noopener nofollow">this http URL</a>

Robotics,Artificial Intelligence,Computer Vision and Pattern Recognition,Machine Learning
Towards Human-Robot Collaborative Surgery: Trajectory and Strategy Learning in Bimanual Peg Transfer

Zhaoyang Jacopo Hu,Ziwei Wang,Yanpei Huang,Aran Sena,Ferdinando Rodriguez y Baena,Etienne Burdet

DOI: https://doi.org/10.1109/lra.2023.3285478

IF: 5.2

2023-08-01

IEEE Robotics and Automation Letters

Abstract:While the traditional control of surgical robots relies on fully manual teleoperations, human-robot collaborative systems promise to address issues such as workspace constrains and laborious tasks. In particular, shared control can reduce the surgeon's workload and improve the overall surgical performance by supporting the surgeon effort during movements while keeping them in charge of complex control phases. In this letter, we propose a segmentation of the bimanual peg transfer task that alternates manual and autonomous control correspondingly. The authority allocation in this shared control framework considers both the limitation of learning-based methods and the higher dexterity of humans during physical interaction. The motion and strategies are transferred from an expert human to a da Vinci Research Kit (dVRK) using an epsilon-greedy on a maximum entropy inverse reinforcement learning algorithm. The model generated enables to train an intelligent agent that can skillfully collaborate with the human operator during the surgical task. The proposed shared control framework is verified both on a virtual platform and then on a real dVRK to assess its usability and robustness. The results show that, compared to traditional teleoperation, our method can achieve faster and more consistent peg transfers. An analysis of the participants' effort also reveals a significantly lower perception of the workload.

robotics
Reinforcement learning of shared control for dexterous telemanipulation: Application to a page turning skill

Takamitsu Matsubara,Takahiro Hasegawa,Kenji Sugimoto

DOI: https://doi.org/10.1109/roman.2015.7333587

2015-08-01

Abstract:The ultimate goal of this study is to develop a method that can accomplish dexterous manipulation of various non-rigid objects by a robotic hand. In this paper, we propose a novel model-free approach using reinforcement learning to learn a shared control policy for dexterous telemanipulation by a human operator. A shared control policy is a probabilistic mapping from the human operator's (master) action and complementary sensor data to the robot (slave) control input for robot actuators. Through the learning process, our method can optimize the shared control policy so that it cooperates to the operator's policy and compensates the lack of sensory information of the operator using complementary sensor data to enhance the dexterity. To validate our method, we adopted a page turning task by telemanipulation and developed an experimental platform with a paper page model and a robot fingertip in simulation. Since the human operator cannot perceive the tactile information of the robot, it may not be as easy as humans do directly. Experimental results suggest that our method is able to learn task-relevant shared control for flexible and enhanced dexterous manipulation by a teleoperated robotic fingertip without tactile feedback to the operator.
A Shared Autonomy Reconfigurable Control Framework for Telemanipulation of Multi-arm Systems

Idil Ozdamar,Marco Laghi,Giorgio Grioli,Arash Ajoudani,Manuel G. Catalano,Antonio Bicchi

DOI: https://doi.org/10.48550/arXiv.2207.09813

2022-07-20

Abstract:Teleoperation is a widely adopted strategy to control robotic manipulators executing complex tasks that require highly dexterous movements and critical high-level intelligence. Classical teleoperation schemes are based on either joystick control, or on more intuitive interfaces which map directly the user arm motions into one robot arm's motions. These approaches have limits when the execution of a given task requires reconfigurable multiple robotic arm systems. Indeed, the simultaneous teleoperation of two or more robot arms could extend the workspace of the manipulation cell, or increase its total payload, or afford other advantages. In different phases of a reconfigurable multi-arm system, each robot could act as an independent arm, or as one of a pair of cooperating arms, or as one of the fingers of a virtual, large robot hand. This manuscript proposes a novel telemanipulation framework that enables both the individual and combined control of any number of robotic arms. Thanks to the designed control architecture, the human operator can intuitively choose the proposed control modalities and the manipulators that make the task convenient to execute through the user interface. Moreover, through the tele-impedance paradigm, the system can address complex tasks that require physical interaction by letting the robot mimic the arm impedance and position references of the human operator. The proposed framework is validated with 8 subjects controlling 4 Franka Emika Panda robots with 7-DoFs to execute a telemanipulation task. Qualitative results of the experiments show us the promising applicability of our framework.

Robotics
ALPHA-$α$ and Bi-ACT Are All You Need: Importance of Position and Force Information/Control for Imitation Learning of Unimanual and Bimanual Robotic Manipulation with Low-Cost System

Masato Kobayashi,Thanpimon Buamanee,Takumi Kobayashi

2024-11-15

Abstract:Autonomous manipulation in everyday tasks requires flexible action generation to handle complex, diverse real-world environments, such as objects with varying hardness and softness. Imitation Learning (IL) enables robots to learn complex tasks from expert demonstrations. However, a lot of existing methods rely on position/unilateral control, leaving challenges in tasks that require force information/control, like carefully grasping fragile or varying-hardness objects. As the need for diverse controls increases, there are demand for low-cost bimanual robots that consider various motor inputs. To address these challenges, we introduce Bilateral Control-Based Imitation Learning via Action Chunking with Transformers(Bi-ACT) and"A" "L"ow-cost "P"hysical "Ha"rdware Considering Diverse Motor Control Modes for Research in Everyday Bimanual Robotic Manipulation (ALPHA-$\alpha$). Bi-ACT leverages bilateral control to utilize both position and force information, enhancing the robot's adaptability to object characteristics such as hardness, shape, and weight. The concept of ALPHA-$\alpha$ is affordability, ease of use, repairability, ease of assembly, and diverse control modes (position, velocity, torque), allowing researchers/developers to freely build control systems using ALPHA-$\alpha$. In our experiments, we conducted a detailed analysis of Bi-ACT in unimanual manipulation tasks, confirming its superior performance and adaptability compared to Bi-ACT without force control. Based on these results, we applied Bi-ACT to bimanual manipulation tasks. Experimental results demonstrated high success rates in coordinated bimanual operations across multiple tasks. The effectiveness of the Bi-ACT and ALPHA-$\alpha$ can be seen through comprehensive real-world experiments. Video available at: <a class="link-external link-https" href="https://mertcookimg.github.io/alpha-biact/" rel="external noopener nofollow">this https URL</a>

Robotics
Learning Variable Compliance Control From a Few Demonstrations for Bimanual Robot with Haptic Feedback Teleoperation System

Tatsuya Kamijo,Cristian C. Beltran-Hernandez,Masashi Hamaya

2024-09-26

Abstract:Automating dexterous, contact-rich manipulation tasks using rigid robots is a significant challenge in robotics. Rigid robots, defined by their actuation through position commands, face issues of excessive contact forces due to their inability to adapt to contact with the environment, potentially causing damage. While compliance control schemes have been introduced to mitigate these issues by controlling forces via external sensors, they are hampered by the need for fine-tuning task-specific controller parameters. Learning from Demonstrations (LfD) offers an intuitive alternative, allowing robots to learn manipulations through observed actions. In this work, we introduce a novel system to enhance the teaching of dexterous, contact-rich manipulations to rigid robots. Our system is twofold: firstly, it incorporates a teleoperation interface utilizing Virtual Reality (VR) controllers, designed to provide an intuitive and cost-effective method for task demonstration with haptic feedback. Secondly, we present Comp-ACT (Compliance Control via Action Chunking with Transformers), a method that leverages the demonstrations to learn variable compliance control from a few demonstrations. Our methods have been validated across various complex contact-rich manipulation tasks using single-arm and bimanual robot setups in simulated and real-world environments, demonstrating the effectiveness of our system in teaching robots dexterous manipulations with enhanced adaptability and safety. Code available at: <a class="link-external link-https" href="https://github.com/omron-sinicx/CompACT" rel="external noopener nofollow">this https URL</a>

Robotics,Artificial Intelligence,Machine Learning
Shared Control of Robot-Robot Collaborative Lifting with Agent Postural and Force Ergonomic Optimization

Lorenzo Rapetti,Yeshasvi Tirupachuri,Alberto Ranavolo,Tomohiro Kawakami,Takahide Yoshiike,Daniele Pucci

DOI: https://doi.org/10.48550/arXiv.2104.13630

2021-04-28

Abstract:Humans show specialized strategies for efficient collaboration. Transferring similar strategies to humanoid robots can improve their capability to interact with other agents, leading the way to complex collaborative scenarios with multiple agents acting on a shared environment. In this paper we present a control framework for robot-robot collaborative lifting. The proposed shared controller takes into account the joint action of both the robots thanks to a centralized controller that communicates with them, and solves the whole-system optimization. Efficient collaboration is ensured by taking into account the ergonomic requirements of the robots through the optimization of posture and contact forces. The framework is validated in an experimental scenario with two iCub humanoid robots performing different payload lifting sequences.

Robotics,Multiagent Systems
Asymmetric Bimanual Control of Dual-arm Serial Manipulator for Robot-assisted Minimally Invasive Surgeries

Hang Su,Yunus Schmirander,Sarah Elena Valderrama-Hincapie,Jairo Pinedo,Xuanyi Zhou,Jiehao Li,Longbin Zhang,Yingbai Hu,Giancarlo Ferrigno,Elena De Momi

DOI: https://doi.org/10.18494/sam.2020.2618

IF: 1.2

2020-01-01

Sensors and Materials

Abstract:Robotic assistance is promising for improving minimally invasive surgery (MIS). This work presents asymmetric bimanual control of a dual-arm serial robot with two remote centers of motion (RCMs) constraints for MIS. In our previous works, general null space controllers to guarantee the fixed RCM constraint have been proposed. However, an incision on a patient's abdominal wall is not fixed owing to the respiration of the patient, which generates an uncertain disturbance at the joints of robotic manipulators. To improve accuracy, a radial basis function neural network is implemented to adapt to these disturbances and control the end-effector position. Finally, the adaptive bimanual control strategy is validated through simulations based on clinical data. The proposed control shows improved accuracy in the end effector position for all the designed surgical tasks. In future works, the algorithm will be validated on an actual dual-arm serial robot making use of a body phantom.
Proactive Robot Control for Collaborative Manipulation Using Human Intent

Zhanibek Rysbek,Siyu Li,Afagh Mehri Shervedani,Milos Zefran

2023-11-06

Abstract:Collaborative manipulation task often requires negotiation using explicit or implicit communication. An important example is determining where to move when the goal destination is not uniquely specified, and who should lead the motion. This work is motivated by the ability of humans to communicate the desired destination of motion through back-and-forth force exchanges. Inherent to these exchanges is also the ability to dynamically assign a role to each participant, either taking the initiative or deferring to the partner's lead. In this paper, we propose a hierarchical robot control framework that emulates human behavior in communicating a motion destination to a human collaborator and in responding to their actions. At the top level, the controller consists of a set of finite-state machines corresponding to different levels of commitment of the robot to its desired goal configuration. The control architecture is loosely based on the human strategy observed in the human-human experiments, and the key component is a real-time intent recognizer that helps the robot respond to human actions. We describe the details of the control framework, and feature engineering and training process of the intent recognition. The proposed controller was implemented on a UR10e robot (Universal Robots) and evaluated through human studies. The experiments show that the robot correctly recognizes and responds to human input, communicates its intent clearly, and resolves conflict. We report success rates and draw comparisons with human-human experiments to demonstrate the effectiveness of the approach.

Robotics

Shared control–based bimanual robot manipulation

Collaborative Bimanual Manipulation Using Optimal Motion Adaptation and Interaction Control

Human–Robot Shared Control Based on Locally Weighted Intent Prediction for a Teleoperated Hydraulic Manipulator System

Brain-Actuated Control of Dual-Arm Robot Manipulation With Relative Motion

An Intent-based Task-aware Shared Control Framework for Intuitive Hands Free Telemanipulation

Robust Shared Control With Stable Contact Servoing for Enhanced Object Transportation by Telerobotic Bimanual Mobile Manipulators

Human-robot Shared Control for Humanoid Manipulator Trajectory Planning

Understanding Shared Control for Assistive Robotic Arms

Bi-Manual Manipulation and Attachment via Sim-to-Real Reinforcement Learning

Shared-Control Teleoperation Paradigms on a Soft Growing Robot Manipulator

A System for Imitation Learning of Contact-Rich Bimanual Manipulation Policies

Shared Control of Bimanual Robotic Limbs With a Brain-Machine Interface for Self-Feeding

PerAct2: Benchmarking and Learning for Robotic Bimanual Manipulation Tasks

Towards Human-Robot Collaborative Surgery: Trajectory and Strategy Learning in Bimanual Peg Transfer

Reinforcement learning of shared control for dexterous telemanipulation: Application to a page turning skill

A Shared Autonomy Reconfigurable Control Framework for Telemanipulation of Multi-arm Systems

ALPHA-$α$ and Bi-ACT Are All You Need: Importance of Position and Force Information/Control for Imitation Learning of Unimanual and Bimanual Robotic Manipulation with Low-Cost System

Learning Variable Compliance Control From a Few Demonstrations for Bimanual Robot with Haptic Feedback Teleoperation System

Shared Control of Robot-Robot Collaborative Lifting with Agent Postural and Force Ergonomic Optimization

Asymmetric Bimanual Control of Dual-arm Serial Manipulator for Robot-assisted Minimally Invasive Surgeries

Proactive Robot Control for Collaborative Manipulation Using Human Intent