Abstract:Background: In this work, we present a novel sensory substitution system that enables to learn three dimensional digital information via touch when vision is unavailable. The system is based on a mouse-shaped device, designed to jointly perceive, with one finger only, local tactile height and inclination cues of arbitrary scalar fields. The device hosts a tactile actuator with three degrees of freedom: elevation, roll and pitch. The actuator approximates the tactile interaction with a plane tangential to the contact point between the finger and the field. Spatial information can therefore be mentally constructed by integrating local and global tactile cues: the actuator provides local cues, whereas proprioception associated with the mouse motion provides the global cues. Methods: The efficacy of the system is measured by a virtual/real object-matching task. Twenty-four gender and age-matched participants (one blind and one blindfolded sighted group) matched a tactile dictionary of virtual objects with their 3D-printed solid version. The exploration of the virtual objects happened in three conditions, i.e., with isolated or combined height and inclination cues. We investigated the performance and the mental cost of approximating virtual objects in these tactile conditions. Results: In both groups, elevation and inclination cues were sufficient to recognize the tactile dictionary, but their combination worked at best. The presence of elevation decreased a subjective estimate of mental effort. Interestingly, only visually impaired participants were aware of their performance and were able to predict it. Conclusions: The proposed technology could facilitate the learning of science, engineering and mathematics in absence of vision, being also an industrial low-cost solution to make graphical user interfaces accessible for people with vision loss.

ALTo: Ad Hoc High-Accuracy Touch Interaction Using Acoustic Localization

UbiTap

VPad: Virtual Writing Tablet for Laptops Leveraging Acoustic Signals

Enable Traditional Laptops with Virtual Writing Capability Leveraging Acoustic Signals.

Device-Free Gesture Tracking Using Acoustic Signals

A Virtual 2D Tactile Array for Soft Actuators Using Acoustic Sensing

Safe Local Navigation for Visually Impaired Users With a Time-of-Flight and Haptic Feedback Device

Mapping Mid-air Haptics with a Low-cost Tactile Robot

Touch Sensors with Overlapping Signals: Concept Investigation on Planar Sensors with Resistive or Optical Transduction

Poster - Sonicnect: Accurate Hands-Free Gesture Input System With Smart Acoustic Sensing

An investigation into the effectiveness of using acoustic touch to assist people who are blind

Real and virtual explorations of the environment and interactive tracking of movable objects for the blind on the basis of tactile-acoustical maps and 3D environment models

An electronic travel aid device to help blind people playing sport

Low-Cost Ultrasonic Range Improvements for an Assistive Device

Tactile Simultaneous Localization and Mapping Using Low-Cost, Wearable LiDAR

Enabling visually impaired people to learn three-dimensional tactile graphics with a 3DOF haptic mouse

Event-Driven Low-Power Gesture Recognition Using Differential Capacitance

A Novel Online Position Estimation Method and Movement Sonification System: The Soniccup

Head mounted device for gesture recognition using acoustic time of flight sensor and 2D camera

Single and bi-layered 2-D acoustic soft tactile skin (AST2)

Semi-autonomous Prosthesis Control Using Minimal Depth Information and Vibrotactile Feedback