Abstract:This paper identifies and addresses the problems with naively combining (reinforcement) learning-based controllers and state estimators for robotic in-hand manipulation. Specifically, we tackle the challenging task of purely tactile, goal-conditioned, dextrous in-hand reorientation with the hand pointing downwards. Due to the limited sensing available, many control strategies that are feasible in simulation when having full knowledge of the object's state do not allow for accurate state estimation. Hence, separately training the controller and the estimator and combining the two at test time leads to poor performance. We solve this problem by coupling the control policy to the state estimator already during training in simulation. This approach leads to more robust state estimation and overall higher performance on the task while maintaining an interpretability advantage over end-to-end policy learning. With our GPU-accelerated implementation, learning from scratch takes a median training time of only 6.5 hours on a single, low-cost GPU. In simulation experiments with the DLR-Hand II and for four significantly different object shapes, we provide an in-depth analysis of the performance of our approach. We demonstrate the successful sim2real transfer by rotating the four objects to all 24 orientations in the $\pi/2$ discretization of SO(3), which has never been achieved for such a diverse set of shapes. Finally, our method can reorient a cube consecutively to nine goals (median), which was beyond the reach of previous methods in this challenging setting.

The Role of Tactile Sensing in Learning and Deploying Grasp Refinement Algorithms

Sim-to-Real Transfer for Robotic Manipulation with Tactile Sensory

Using Tactile Sensing to Improve the Sample Efficiency and Performance of Deep Deterministic Policy Gradients for Simulated In-Hand Manipulation Tasks

Tactile Regrasp: Grasp Adjustments via Simulated Tactile Transformations

Fine Manipulation Using a Tactile Skin: Learning in Simulation and Sim-to-Real Transfer

See to Touch: Learning Tactile Dexterity through Visual Incentives

Tactile Active Inference Reinforcement Learning for Efficient Robotic Manipulation Skill Acquisition

More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch

Estimator-Coupled Reinforcement Learning for Robust Purely Tactile In-Hand Manipulation

Center-of-Mass-Based Regrasping of Unknown Objects Using Reinforcement Learning and Tactile Sensing.

DexTouch: Learning to Seek and Manipulate Objects with Tactile Dexterity

Learning In-Hand Translation Using Tactile Skin With Shear and Normal Force Sensing

Integrating High-Resolution Tactile Sensing into Grasp Stability Prediction

Bi-Touch: Bimanual Tactile Manipulation With Sim-to-Real Deep Reinforcement Learning

Learning to Regrasp Using Visual–Tactile Representation-Based Reinforcement Learning

Sim-to-Real Model-Based and Model-Free Deep Reinforcement Learning for Tactile Pushing

Dexterous In-Hand Manipulation of Slender Cylindrical Objects through Deep Reinforcement Learning with Tactile Sensing

Learning Tactile Insertion in the Real World

Sim2Real Manipulation on Unknown Objects with Tactile-based Reinforcement Learning

Rotating without Seeing: Towards In-hand Dexterity through Touch

Autonomous Learning of Page Flipping Movements via Tactile Feedback