Abstract:Abstract Background Hand amputation can have a truly debilitating impact on the life of the affected person. A multifunctional myoelectric prosthesis controlled using pattern classification can be used to restore some of the lost motor abilities. However, learning to control an advanced prosthesis can be a challenging task, but virtual and augmented reality (AR) provide means to create an engaging and motivating training. Methods In this study, we present a novel training framework that integrates virtual elements within a real scene (AR) while allowing the view from the first-person perspective. The framework was evaluated in 13 able-bodied subjects and a limb-deficient person divided into intervention (IG) and control (CG) groups. The IG received training by performing simulated clothespin task and both groups conducted a pre- and posttest with a real prosthesis. When training with the AR, the subjects received visual feedback on the generated grasping force. The main outcome measure was the number of pins that were successfully transferred within 20 min (task duration), while the number of dropped and broken pins were also registered. The participants were asked to score the difficulty of the real task (posttest), fun-factor and motivation, as well as the utility of the feedback. Results The performance (median/interquartile range) consistently increased during the training sessions (4/3 to 22/4). While the results were similar for the two groups in the pretest, the performance improved in the posttest only in IG. In addition, the subjects in IG transferred significantly more pins (28/10.5 versus 14.5/11), and dropped (1/2.5 versus 3.5/2) and broke (5/3.8 versus 14.5/9) significantly fewer pins in the posttest compared to CG. The participants in IG assigned (mean ± std) significantly lower scores to the difficulty compared to CG (5.2 ± 1.9 versus 7.1 ± 0.9), and they highly rated the fun factor (8.7 ± 1.3) and usefulness of feedback (8.5 ± 1.7). Conclusion The results demonstrated that the proposed AR system allows for the transfer of skills from the simulated to the real task while providing a positive user experience. The present study demonstrates the effectiveness and flexibility of the proposed AR framework. Importantly, the developed system is open source and available for download and further development.

Guide Dog AR: A Tactile and Auditory Assisting Device Design with the Motif of a Guide Dog for the Visually Impaired

Hybrid auditory feedback: a new method for mobility assistance of the visually impaired.

Right mix of speech and non-speech: hybrid auditory feedback in mobility assistance of the visually impaired

Design of Electronic Guide Dog Based on User Experience

See Spot Guide: Accessible Interfaces for an Assistive Quadruped Robot

Teleguidance-based remote navigation assistance for visually impaired and blind people-usability and user experience

The Aerial Guide Dog: A Low-Cognitive-Load Indoor Electronic Travel Aid for Visually Impaired Individuals

"The Guide Has Your Back": Exploring How Sighted Guides Can Enhance Accessibility in Social Virtual Reality for Blind and Low Vision People

System Configuration and Navigation of a Guide Dog Robot: Toward Animal Guide Dog-Level Guiding Work

Cumulative Study and Development of Obstacle Alert and Assistance Device for Visually Impaired People Using Machine Learning

Augmented reality: a view to future visual supports for people with disability

Holographic Hintways: A systems feasibility and usability study of augmented reality cueing for gait adaptation

Lessons Learned from Developing a Human-Centered Guide Dog Robot for Mobility Assistance

Guide dog versus robot dog: assembling visually impaired people with non-human agents and achieving assisted mobility through distributed co-constructed perception

Multi-Section Sensing and Vibrotactile Perception for Walking Guide of Visually Impaired Person

Towards Robotic Companions: Understanding Handler-Guide Dog Interactions for Informed Guide Dog Robot Design

Virtual Guide Dog: Next Generation Pedestrian Signal for the Visually Impaired

An AI Guide to Enhance Accessibility of Social Virtual Reality for Blind People

Immersive augmented reality system for the training of pattern classification control with a myoelectric prosthesis

Heart Seeker – A Guide Robot with Warmth

A wearable assistive system for the visually impaired using object detection, distance measurement and tactile presentation