Abstract:Background: Hearing loss is becoming a real social and health problem. Its prevalence in the elderly is an epidemic. The risk of developing hearing loss is also growing among younger people. If left untreated, hearing loss can perpetuate development of neurodegenerative diseases, including dementia. Despite recent advancements in hearing aid (HA) and cochlear implant (CI) technologies, hearing impaired users still encounter significant practical and social challenges, with or without aids. In particular, they all struggle with understanding speech in challenging acoustic environments, especially in presence of a competing speaker. Objectives: In the current proof-of-concept study we tested whether multisensory stimulation, pairing audition and a minimal-size touch device would improve intelligibility of speech in noise. Methods: To this aim we developed an audio-to-tactile sensory substitution device (SSD) transforming low-frequency speech signals into tactile vibrations delivered on two finger tips. Based on the inverse effectiveness law, i.e., multisensory enhancement is strongest when signal-to-noise ratio is lowest between senses, we embedded non-native language stimuli in speech-like noise and paired it with a low-frequency input conveyed through touch. Results: We found immediate and robust improvement in speech recognition (i.e. in the Signal-To-Noise-ratio) in the multisensory condition without any training, at a group level as well as in every participant. The reported improvement at the group-level of 6 dB was indeed major considering that an increase of 10 dB represents a doubling of the perceived loudness. Conclusions: These results are especially relevant when compared to previous SSD studies showing effects in behavior only after a demanding cognitive training. We discuss the implications of our results for development of SSDs and of specific rehabilitation programs for the hearing impaired either using or not using HAs or CIs. We also discuss the potential application of such a set-up for sense augmentation, such as when learning a new language.

Microprocessor-based hearing aid for the deaf

Right mix of speech and non-speech: hybrid auditory feedback in mobility assistance of the visually impaired

A Wearable Vision-To-Audio Sensory Substitution Device for Blind Assistance and the Correlated Neural Substrates

Speech visualization by integrating features for the hearing impaired

A Multisensory Approach to Present Phonemes as Language Through a Wearable Haptic Device

Improved tactile speech perception using audio-to-tactile sensory substitution with formant frequency focusing

Immediate improvement of speech-in-noise perception through multisensory stimulation via an auditory to tactile sensory substitution

Skin-Integrated Wireless Odor Message Delivery Electronics for the Deaf-blind

Improving speech perception for hearing-impaired listeners using audio-to-tactile sensory substitution with multiple frequency channels

A multi-channel hearing prosthesis for profound-to-total hearing loss

Audiovisual integration system based on synesthesia for people with impaired hearing

A Microphone Array and Voice Algorithm based Smart Hearing Aid

Mobile Sound Recognition for the Deaf and Hard of Hearing

Perceiving Graphical and Pictorial Information via Hearing and Touch

Simultaneous Multimodal PC Access for People With Disabilities by Integrating Head Tracking, Speech Recognition, and Tongue Motion

GLOS: GLOve for Speech Recognition

Improved tactile speech robustness to background noise with a dual-path recurrent neural network noise-reduction method

A dual-mode human computer interface combining speech and tongue motion for people with severe disabilities

Attention-Augmented Electromagnetic Representation of Sign Language for Human-Computer Interaction in Deaf-and-Mute Community

Bidirectional Braille-speech Communication System for Deafblind Students

Three-Dimensional Encoding Approach for Wearable Tactile Communication Devices