Abstract:This study proposes a new hybrid multi-modal sensory feedback system for prosthetic hands that can provide not only haptic and proprioceptive feedback but also facilitate object recognition without the aid of vision. Modality-matched haptic perception was provided using a mechanotactile feedback system that can proportionally apply the gripping force through the use of a force controller. A vibrotactile feedback system was also employed to distinguish four discrete grip positions of the prosthetic hand. The system performance was evaluated with a total of 32 participants in three different experiments (i) haptic feedback, (ii) proprioceptive feedback and (iii) object recognition with hybrid haptic-proprioceptive feedback. The results from the haptic feedback experiment showed that the participants' ability to accurately perceive applied force depended on the amount of force applied. As the feedback force was increased, the participants tended to underestimate the force levels, with a decrease in the percentage of force estimation. Of the three arm locations (forearm volar, forearm ventral and bicep), and two muscle states (relaxed and tensed) tested, the highest accuracy was obtained for the bicep location in the relaxed state. The results from the proprioceptive feedback experiment showed that participants could very accurately identify four different grip positions of the hand prosthesis (i.e., open hand, wide grip, narrow grip, and closed hand) without a single case of misidentification. In experiment 3, participants could identify objects with different shapes and stiffness with an overall high success rate of 90.5% across all combinations of location and muscle state. The feedback location and muscle state did not have a significant effect on object recognition accuracy. Overall, our study results indicate that the hybrid feedback system may be a very effective way to enrich a prosthetic hand user's experience of the stiffness and shape of commonly manipulated objects.

Perceptual Issues of a Passive Haptics Feedback Based MR System

Passive Haptics Based MR System for Geography Teaching

Multi-modal Sensing-Based Interactive Glove System for Teleoperation and VR/AR

A visuo-haptic mixed reality manual milling training simulator

Haptic Feedback Helps Me? A VR-SAR Remote Collaborative System with Tangible Interaction

Peripersonal Space Tele-Operation in Virtual Reality: The Role of Tactile - Force Feedback

Evaluation of Direct Physics-Inspired Interaction for Mixed Reality Based on Optical See-Through Head-Mounted Displays

Enhanced Wearable Force-Feedback Mechanism for Free-Range Haptic Experience Extended by Pass-Through Mixed Reality

Instilling the perception of weight in augmented reality using minimal haptic feedback

Haptic Sensing and Feedback Techniques toward Virtual Reality

Evaluation of Hand-Based Interaction for Near-Field Mixed Reality with Optical See-Through Head-Mounted Displays

Evaluation of Point-to-Point Reaching Performance in Mixed Reality and Virtual Reality

A Proof of Concept Integrated Multi-systems Approach for Large Scale Tactile Feedback in VR

Depth-aware Interactive Display Method for Vision-Tangible Mixed Reality

Design and Characterisation of Multi-cavity, Fluidic Haptic Feedback System for Mechano-tactile Feedback

Effects of Multisensory Feedback on the Perception and Performance of Virtual Reality Hand-Retargeted Interaction

Design and Research of Multimodal Fusion Feedback Device Based on Virtual Interactive System

Physics-Inspired Input Method For Near-Field Mixed Reality Applications Using Latent Active Correction

A Mixed Reality Training System for Hand-Object Interaction in Simulated Microgravity Environments

Experimental Evaluation of a Hybrid Sensory Feedback System for Haptic and Kinaesthetic Perception in Hand Prostheses

Haptic display for virtual reality: progress and challenges.