Abstract:In crowded settings, the human brain can focus on speech from a target speaker, given prior knowledge of how they sound. We introduce a novel intelligent hearable system that achieves this capability, enabling target speech hearing to ignore all interfering speech and noise, but the target speaker. A naive approach is to require a clean speech example to enroll the target speaker. This is however not well aligned with the hearable application domain since obtaining a clean example is challenging in real world scenarios, creating a unique user interface problem. We present the first enrollment interface where the wearer looks at the target speaker for a few seconds to capture a single, short, highly noisy, binaural example of the target speaker. This noisy example is used for enrollment and subsequent speech extraction in the presence of interfering speakers and noise. Our system achieves a signal quality improvement of 7.01 dB using less than 5 seconds of noisy enrollment audio and can process 8 ms of audio chunks in 6.24 ms on an embedded CPU. Our user studies demonstrate generalization to real-world static and mobile speakers in previously unseen indoor and outdoor multipath environments. Finally, our enrollment interface for noisy examples does not cause performance degradation compared to clean examples, while being convenient and user-friendly. Taking a step back, this paper takes an important step towards enhancing the human auditory perception with artificial intelligence. We provide code and data at: <a class="link-external link-https" href="https://github.com/vb000/LookOnceToHear" rel="external noopener nofollow">this https URL</a>.

Environmental noise reduction based on speech/non-speech identification for hearing aids

Deep Neural Network Based Noised Asian Speech Enhancement and Its Implementation on a Hearing Aid App.

Threshold-Based Noise Detection and Reduction for Automatic Speech Recognition System in Human-Robot Interactions

Deep Neural Network Based Environment Sound Classification and Its Implementation on Hearing Aid App

Sub-band Adaptive Noise Reduction Algorithm to Improve Speech Intelligibility

Restoring speech intelligibility for hearing aid users with deep learning

Improved tactile speech robustness to background noise with a dual-path recurrent neural network noise-reduction method

Adaptive Noise Reduction Algorithm Based on SPP and NMF for Environmental Sound Event Recognition under Low-SNR Conditions

High-Fidelity Noise Reduction with Differentiable Signal Processing

Improved Speech Intelligibility in Noise with a Single-Microphone Noise Reduction Technique

Environment-aware Reconfigurable Noise Suppression

Deep Denoising for Hearing Aid Applications

Environmental noise reduction system using fuzzy neural network and adaptive fuzzy algorithms

Noise reduction in speech signals using adaptive independent component analysis (ICA) for hands free communication devices

An Automated Hearing Test Equipment Based on Active Noise Control Technology.

Speech Dereverberation and Noise Reduction for both diffusive noise field and point noise source in Binaural Hearing Aids: Preliminary Version

Look Once to Hear: Target Speech Hearing with Noisy Examples

Improving Speech Intelligibility of Severely Hearing-Impaired People by Frequency-Lowering Technique

A Multiband Noise Reduction Wiener Filter Algorithm for Hearing Aid

Using deep learning to improve the intelligibility of a target speaker in noisy multi-talker environments for people with normal hearing and hearing loss

Automatic bioacoustics noise reduction method based on a deep feature loss network