Abstract:Vision is the main component of current robotics systems that is used for manipulating objects. However, solely relying on vision for hand-object pose tracking faces challenges such as occlusions and objects moving out of view during robotic manipulation. In this work, we show that object kinematics can be inferred from local haptic feedback at the robot-object contact points, combined with robot kinematics information given an initial vision estimate of the object pose. A planar, dual-arm, teleoperated robotic setup was built to manipulate an object with hands shaped like circular discs. The robot hands were built with rubber cladding to allow for rolling contact without slipping. During stable grasping by the dual arm robot, under quasi-static conditions, the surface of the robot hand and object at the contact interface is defined by local geometric constraints. This allows one to define a relation between object orientation and robot hand orientation. With rolling contact, the displacement of the contact point on the object surface and the hand surface must be equal and opposite. This information, coupled with robot kinematics, allows one to compute the displacement of the object from its initial location. The mathematical formulation of the geometric constraints between robot hand and object is detailed. This is followed by the methodology in acquiring data from experiments to compute object kinematics. The sensors used in the experiments, along with calibration procedures, are presented before computing the object kinematics from recorded haptic feedback. Results comparing object kinematics obtained purely from vision and from haptics are presented to validate our method, along with the future ideas for perception via haptic manipulation.

Integrating vision, haptics and proprioception into a feedback controller for in-hand manipulation of unknown objects

Adaptive Grasping of the Soft Prosthetic Hand Via Tactile Feedback

Dexterous Manoeuvre Through Touch in a Cluttered Scene

Beyond End-Effector: Utilizing High-Resolution Tactile Signals for Physical Human–Robot Interaction

Towards Haptic-Based Dual-Arm Manipulation

Tactile-Driven Non-Prehensile Object Manipulation via Extrinsic Contact Mode Control

Integrating and Evaluating Visuo-tactile Sensing with Haptic Feedback for Teleoperated Robot Manipulation

Adaptive Fingers Coordination for Robust Grasp and In-Hand Manipulation Under Disturbances and Unknown Dynamics

Learning Haptic-Based Object Pose Estimation for In-Hand Manipulation Control With Underactuated Robotic Hands

Dexterous Soft Hands Linearize Feedback-Control for In-Hand Manipulation

Control framework for dexterous manipulation using dynamic visual servoing and tactile sensors' feedback

Adaptive Motion Planning for Multi-fingered Functional Grasp via Force Feedback

Robust Visual Feedback Control for Precise In-Hand Manipulation Using Parallel Soft Actuators

Sensorimotor-inspired Tactile Feedback and Control Improve Consistency of Prosthesis Manipulation in the Absence of Direct Vision

TEXterity -- Tactile Extrinsic deXterity: Simultaneous Tactile Estimation and Control for Extrinsic Dexterity

A new control strategy for DLR's multisensory articulated hand

Tactile Dexterity: Manipulation Primitives with Tactile Feedback

Robust In-Hand Manipulation with Extrinsic Contacts

Contact-Implicit Model Predictive Control for Dexterous In-hand Manipulation: A Long-Horizon and Robust Approach

Multifingered Robot Hand Compliant Manipulation Based on Vision-Based Demonstration and Adaptive Force Control

Simultaneous Tactile Estimation and Control of Extrinsic Contact