Abstract:Background: Hearing loss is becoming a real social and health problem. Its prevalence in the elderly is an epidemic. The risk of developing hearing loss is also growing among younger people. If left untreated, hearing loss can perpetuate development of neurodegenerative diseases, including dementia. Despite recent advancements in hearing aid (HA) and cochlear implant (CI) technologies, hearing impaired users still encounter significant practical and social challenges, with or without aids. In particular, they all struggle with understanding speech in challenging acoustic environments, especially in presence of a competing speaker. Objectives: In the current proof-of-concept study we tested whether multisensory stimulation, pairing audition and a minimal-size touch device would improve intelligibility of speech in noise. Methods: To this aim we developed an audio-to-tactile sensory substitution device (SSD) transforming low-frequency speech signals into tactile vibrations delivered on two finger tips. Based on the inverse effectiveness law, i.e., multisensory enhancement is strongest when signal-to-noise ratio is lowest between senses, we embedded non-native language stimuli in speech-like noise and paired it with a low-frequency input conveyed through touch. Results: We found immediate and robust improvement in speech recognition (i.e. in the Signal-To-Noise-ratio) in the multisensory condition without any training, at a group level as well as in every participant. The reported improvement at the group-level of 6 dB was indeed major considering that an increase of 10 dB represents a doubling of the perceived loudness. Conclusions: These results are especially relevant when compared to previous SSD studies showing effects in behavior only after a demanding cognitive training. We discuss the implications of our results for development of SSDs and of specific rehabilitation programs for the hearing impaired either using or not using HAs or CIs. We also discuss the potential application of such a set-up for sense augmentation, such as when learning a new language.

Assistive Magnetic Skin System for Speech Reconstruction: An Empowering Technology for Aphonic Individuals

A Wearable Vision-To-Audio Sensory Substitution Device for Blind Assistance and the Correlated Neural Substrates

Attention Bidirectional LSTM Networks Based Mime Speech Recognition Using Semg Data

Decoding Silent Speech Commands from Articulatory Movements Through Soft Magnetic Skin and Machine Learning

A high-performance neuroprosthesis for speech decoding and avatar control

Wearable intelligent throat enables natural speech in stroke patients with dysarthria

Electromyogram-Based Lip-Reading via Unobtrusive Dry Electrodes and Machine Learning Methods.

An accurate and rapidly calibrating speech neuroprosthesis

Advancing Voice Biometrics for Dysarthria Speakers Using Multitaper LFCC and Voice Conversion Data Augmentation

A high-performance speech neuroprosthesis

All-weather, natural silent speech recognition via machine-learning-assisted tattoo-like electronics

A Multisensory Approach to Present Phonemes as Language Through a Wearable Haptic Device

Immediate improvement of speech-in-noise perception through multisensory stimulation via an auditory to tactile sensory substitution

The speech neuroprosthesis

Speaking without vocal folds using a machine-learning-assisted wearable sensing-actuation system

Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis

Automatic Speech Recognition and its Visual Perception Via a Cymatics Based Display

Using Voice Technologies to Support Disabled People

Magnetized Microcilia Array‐Based Self‐Powered Electronic Skin for Micro‐Scaled 3D Morphology Recognition and High‐capacity Communication

Electromyogram-strain synergetic intelligent artificial throat

Geometry of orofacial neuromuscular signals: speech articulation decoding using surface electromyography