Abstract:Employing skin-like tactile sensors on robots enhances both the safety and usability of collaborative robots by adding the capability to detect human contact. Unfortunately, simple binary tactile sensors alone cannot determine the context of the human contact -- whether it is a deliberate interaction or an unintended collision that requires safety manoeuvres. Many published methods classify discrete interactions using more advanced tactile sensors or by analysing joint torques. Instead, we propose to augment the intention recognition capabilities of simple binary tactile sensors by adding a robot-mounted camera for human posture analysis. Different interaction characteristics, including touch location, human pose, and gaze direction, are used to train a supervised machine learning algorithm to classify whether a touch is intentional or not with an F1-score of 86%. We demonstrate that multimodal intention recognition is significantly more accurate than monomodal analyses with the collaborative robot Baxter. Furthermore, our method can also continuously monitor interactions that fluidly change between intentional or unintentional by gauging the user's attention through gaze. If a user stops paying attention mid-task, the proposed intention and attention recognition algorithm can activate safety features to prevent unsafe interactions. We also employ a feature reduction technique that reduces the number of inputs to five to achieve a more generalized low-dimensional classifier. This simplification both reduces the amount of training data required and improves real-world classification accuracy. It also renders the method potentially agnostic to the robot and touch sensor architectures while achieving a high degree of task adaptability.

Cross-Modal Reconstruction for Tactile Signal in Human-Robot Interaction

A Vision-Based Tactile Sensing System for Multimodal Contact Information Perception via Neural Network

Toward General Cross-Modal Signal Reconstruction for Robotic Teleoperation

A Cross-Modal Tactile Sensor Design for Measuring Robotic Grasping Forces

Kinematics-based 3D Human-Object Interaction Reconstruction from Single View

Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensing

Robust Learning of Tactile Force Estimation through Robot Interaction

Attention for Robot Touch: Tactile Saliency Prediction for Robust Sim-to-Real Tactile Control

TactileAR: Active Tactile Pattern Reconstruction

ACROSS: A Deformation-Based Cross-Modal Representation for Robotic Tactile Perception

Interpreting and Predicting Tactile Signals for the SynTouch BioTac

Bridging vision and touch: advancing robotic interaction prediction with self-supervised multimodal learning

Lifelong Visual-Tactile Cross-Modal Learning for Robotic Material Perception

Vision- and tactile-based continuous multimodal intention and attention recognition for safer physical human-robot interaction

Using tactile sensation for learning contact knowledge: Discriminate collision from physical interaction

Imagine2touch: Predictive Tactile Sensing for Robotic Manipulation using Efficient Low-Dimensional Signals

Adaptive visual–tactile fusion recognition for robotic operation of multi-material system

Human-Robot Co-Carrying Using Visual and Force Sensing

Haptic Signal Reconstruction for Cross-Modal Communications

A Solution of Human-Computer Remote Interaction with Tactile Feedback

Leveraging Compliant Tactile Perception for Haptic Blind Surface Reconstruction