Abstract:Dexterous manipulation concerns the control of a robot hand to manipulate an object in a desired manner. While classical dexterous manipulation strategies are based on stable grasping (or force closure), many human-like manipulation tasks do not maintain grasp stability and often utilize the dynamics of the object rather than the closed form of kinematic relation between the object and the robotic hand. Such manipulation strategies are referred as nonprehensile or dynamic dexterous manipulation in the literature. Nonprehensile manipulation often involves fast and agile movements such as throwing and flipping. Due to the complexity of such motions and uncertainties associated with them, it has been challenging to realize nonprehensile manipulation tasks in a reliable way. In this paper, we propose a new control strategy to realize practical nonprehensile manipulation. First, we make explicit use of multiple modalities of sensory data for the design of control law. Specifically, force data are employed for feedforward control, while position data are used for feedback control. Secondly, control signals (both feedback and feedforward) are obtained through multisensory learning from demonstration (LfD) experiments designed and performed for specific nonprehensile manipulation tasks of concern. To prove the concept of the proposed control strategy, experimental tests were conducted for a dynamic spinning task using a sensory-rich, two-finger robotic hand. The control performance (i.e., the speed and accuracy of the spinning task) was also compared with that of classical dexterous manipulation based on force closure and finger gaiting.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to overcome the limitations of traditional model - based methods in achieving non - prehensile dexterous manipulation by learning from the demonstration data of human experts, so as to achieve faster and more agile non - prehensile manipulation. Specifically, the author proposes a new control strategy, which uses multi - modal sensing data (such as visual, force and position data) for learning from demonstration (LfD) to design control laws. This method aims to improve the control performance of dynamic rotation tasks, that is, the speed and accuracy of rotating objects, and is compared with traditional dexterous manipulation methods based on force closure and finger stepping. The key points of the paper are as follows: 1. **Non - prehensile Manipulation**: Different from traditional dexterous manipulation relying on stable grasping, non - prehensile manipulation does not maintain a stable grasp, but uses the intrinsic dynamic characteristics of the object to complete the task, such as rapid rotation, throwing and flipping. 2. **Multi - modal Sensing Data**: Use a variety of sensor data (including force sensors, position encoders and visual sensors) to design control laws, where force data is used for feed - forward control and position data is used for feedback control. 3. **Learning from Demonstration**: Through demonstration experiments of specific non - prehensile manipulation tasks, obtain control signals (including feedback and feed - forward signals), which are used to generate reference trajectories. 4. **Experimental Verification**: Through a dynamic rotation task, an experimental test is carried out using a two - fingered robot hand equipped with multiple sensors to verify the effectiveness of the proposed control strategy. In short, the main contribution of this paper is to propose a systematic method. By learning from the demonstration data of human experts, reliable reference trajectories are generated. Combined with multi - modal sensing data, efficient and flexible non - prehensile dexterous manipulation is achieved. This provides new ideas and methods for the autonomous operation of robots in complex environments.

Nonprehensile Manipulation for Rapid Object Spinning via Multisensory Learning from Demonstration

Dexterous Manoeuvre Through Touch in a Cluttered Scene

Tactile-Driven Non-Prehensile Object Manipulation via Extrinsic Contact Mode Control

Learning Dexterous Manipulation Policies from Experience and Imitation

Contact-Implicit Model Predictive Control for Dexterous In-hand Manipulation: A Long-Horizon and Robust Approach

Mechanical Intelligence for Prehensile In-Hand Manipulation of Spatial Trajectories.

Non-Parametric Self-Identification and Model Predictive Control of Dexterous In-Hand Manipulation

Deep Dynamics Models for Learning Dexterous Manipulation

A new control strategy for DLR's multisensory articulated hand

Learning thin deformable object manipulation with a multi-sensory integrated soft hand

Dexterous Soft Hands Linearize Feedback-Control for In-Hand Manipulation

Learning Forceful Manipulation Skills from Multi-modal Human Demonstrations

A Survey of Multifingered Robotic Manipulation: Biological Results, Structural Evolvements, and Learning Methods

Design and Control of Roller Grasper V2 for In-Hand Manipulation

Toward Fine Contact Interactions: Learning to Control Normal Contact Force with Limited Information

Sampling-Based Model Predictive Control for Dexterous Manipulation on a Biomimetic Tendon-Driven Hand

Multifingered Robot Hand Compliant Manipulation Based on Vision-Based Demonstration and Adaptive Force Control

Adaptive Motion Planning for Multi-fingered Functional Grasp via Force Feedback

Reinforcement learning of shared control for dexterous telemanipulation: Application to a page turning skill

Contact Optimization with Learning from Demonstration: Application in Long-term Non-prehensile Planar Manipulation

Perception, Control and Hardware for In-Hand Slip-Aware Object Manipulation with Parallel Grippers