Abstract:Over 1.5 billion people worldwide live with hearing impairment. Despite various technologies that have been created for individuals with such disabilities, most of these technologies are either extremely expensive or inaccessible for everyday use in low-medium income countries. In order to combat this issue, we have developed a new assistive device, EchoVest, for blind/deaf people to intuitively become more aware of their environment. EchoVest transmits vibrations to the user's body by utilizing transcutaneous electric nerve stimulation (TENS) based on the source of the sounds. EchoVest also provides various features, including sound localization, sound classification, noise reduction, and depth perception. We aimed to outperform CNN-based machine-learning models, the most commonly used machine learning model for classification tasks, in accuracy and computational costs. To do so, we developed and employed a novel audio pipeline that adapts the Audio Spectrogram Transformer (AST) model, an attention-based model, for our sound classification purposes, and Fast Fourier Transforms for noise reduction. The application of Otsu's Method helped us find the optimal thresholds for background noise sound filtering and gave us much greater accuracy. In order to calculate direction and depth accurately, we applied Complex Time Difference of Arrival algorithms and SOTA localization. Our last improvement was to use blind source separation to make our algorithms applicable to multiple microphone inputs. The final algorithm achieved state-of-the-art results on numerous checkpoints, including a 95.7\% accuracy on the ESC-50 dataset for environmental sound classification.

What problem does this paper attempt to address?

The main problem this paper attempts to address is providing a low-cost and easy-to-use assistive device for individuals with hearing impairments to enhance their environmental awareness in daily life. Specifically, the paper introduces EchoVest, a wearable device that converts sound information into body vibrations through Transcutaneous Electrical Nerve Stimulation (TENS) technology, aimed at helping deaf-blind individuals better perceive their surroundings. ### Main Problems: 1. **High Cost**: Existing hearing aids such as hearing aids and cochlear implants are expensive, making them unaffordable for people in many low- and middle-income countries. 2. **Insufficient Functionality**: Current technologies have shortcomings in sound source localization, depth perception, and noise suppression, failing to provide comprehensive environmental awareness. 3. **Poor Accessibility**: Existing hearing aids require complex customization and medical evaluation, making them difficult for ordinary users to obtain. ### Solution: - **EchoVest**: A low-cost wearable device that uses TENS technology to convert sound information into body vibrations, helping users perceive the direction and distance of sounds. - **Sound Source Localization and Depth Perception**: Achieved through a multi-microphone array and complex algorithms (such as TDoA and GCC-PHAT) for high-precision sound source localization and depth perception. - **Noise Suppression**: Effectively removes background noise by combining Fast Fourier Transform (FFT) and Otsu's Method, improving the accuracy of sound classification. - **Machine Learning**: Uses the Audio Spectrogram Transformer (AST) model for sound classification, achieving an accuracy of 95.7% on the ESC-50 dataset. ### Objectives: - **Improve Accessibility**: The manufacturing cost of EchoVest is only $98.90, much lower than existing hearing aids, making it more accessible to people in low- and middle-income countries. - **Enhance Environmental Awareness**: Through multidimensional vibration feedback, it helps users better perceive their surroundings, improving their quality of life. - **Real-time Performance**: Ensures the device operates efficiently in real-time environments, providing instant environmental awareness information. Through these methods, EchoVest aims to provide an affordable and powerful assistive tool for individuals with hearing impairments, improving their quality of daily life.

EchoVest: Real-Time Sound Classification and Depth Perception Expressed through Transcutaneous Electrical Nerve Stimulation

A Wearable Vision-To-Audio Sensory Substitution Device for Blind Assistance and the Correlated Neural Substrates

Automatic Auditory Streaming Restores Missing Temporal Modulations in Echoic Speech

Help the Blind See: Assistance for the Visually Impaired through Augmented Acoustic Simulation

EchoSee: An Assistive Mobile Application for Real-Time 3D Environment Reconstruction and Sonification Supporting Enhanced Navigation for People with Vision Impairments

Deep learning-based auditory attention decoding in listeners with hearing impairment

DeepVQE: Real Time Deep Voice Quality Enhancement for Joint Acoustic Echo Cancellation, Noise Suppression and Dereverberation

EarEcho

Objective speech intelligibility prediction using a deep learning model with continuous speech-evoked cortical auditory responses

Real-time Neonatal Chest Sound Separation using Deep Learning

DeepBeat: A multi-task deep learning approach to assess signal quality and arrhythmia detection in wearable devices

Enhancing Situational Awareness with VAS-Compass Net for the Recognition of Directional Vehicle Alert Sounds

LIDAR Assist Spatial Sensing for the Visually Impaired and Performance Analysis

An investigation into the effectiveness of using acoustic touch to assist people who are blind

A Wearable Assistant Device for the Hearing Impaired to Recognize Emergency Vehicle Sirens with Edge Computing

Tuning In: Analysis of Audio Classifier Performance in Clinical Settings with Limited Data

Decoding Envelope and Frequency-Following EEG Responses to Continuous Speech Using Deep Neural Networks

ActSonic: Recognizing Everyday Activities from Inaudible Acoustic Wave Around the Body

Cumulative Study and Development of Obstacle Alert and Assistance Device for Visually Impaired People Using Machine Learning

EarSpeech: Exploring In-Ear Occlusion Effect on Earphones for Data-efficient Airborne Speech Enhancement

Neural decoding of attentional selection in multi-speaker environments without access to clean sources