Abstract:A cochlear implant (CI) is a surgically implanted electronic device that partially restores hearing to people suffering from profound hearing loss. Although CI users, in general, obtain a very good reception of continuous speech in the absence of background noise, they face severe limitations in the context of music perception and appreciation. The main reasons for these limitations are related to channel interactions created by the broad spread of electrical fields in the cochlea and to the low number of electrodes that stimulate it. Moreover, CIs have severe limitations when it comes to transmitting the temporal fine structure of acoustic signals, and hence, these devices elicit poor pitch and timber perception. For these reasons, several signal processing algorithms have been proposed to make music more accessible for CI users, trying to reduce the complexity of music signals or remixing them to enhance certain components, such as the lead singing voice. In this work, a deep neural network that performs real-time audio source separation to remix music for CI users is presented. The implementation is based on multi-layer perception (MLP) and has been evaluated using objective instrumental measurements to ensure clean source estimation. Furthermore, experiments in 10 normal hearing (NH) and 13 CI users to investigate how the vocals to instruments ratio (VIR) set by the tested listeners were affected in realistic environments with and without visual information. The objective instrumental results fulfill the benchmark reported in previous studies by introducing distortions that are shown to not be perceived by CI users. Moreover, the implemented model was optimized to perform real-time source separation. The experimental results show that CI users prefer vocals 8 dB enhanced with the respect to the instruments independent of acoustic sound scenarios and visual information. In contrast, NH listeners did not prefer a VIR different than zero dB.

Remixing Music for Hearing Aids Using Ensemble of Fine-Tuned Source Separators

The ICASSP SP Cadenza Challenge: Music Demixing/Remixing for Hearing Aids

The first Cadenza challenges: using machine learning competitions to improve music for listeners with a hearing loss

The USTC System for Cadenza 2024 Challenge

The First Cadenza Signal Processing Challenge: Improving Music for Those With a Hearing Loss

Music Enhancement with Deep Filters: A Technical Report for The ICASSP 2024 Cadenza Challenge

Design and Evaluation of a Real-Time Audio Source Separation Algorithm to Remix Music for Cochlear Implant Users

A versatile deep-neural-network-based music preprocessing and remixing scheme for cochlear implant listeners

Enhancing music recognition using deep learning-powered source separation technology for cochlear implant users

Remixing Preferences for Western Instrumental Classical Music of Bilateral Cochlear Implant Users

Benchmarks and leaderboards for sound demixing tasks

A Low-Latency Hybrid Multi-Channel Speech Enhancement System for Hearing Aids

Binaural Audio Source Remixing with Microphone Array Listening Devices

A subjective evaluation of different music preprocessing approaches in cochlear implant listeners

HAAQI-Net: A Non-intrusive Neural Music Audio Quality Assessment Model for Hearing Aids

Deep Neural Network Based Noised Asian Speech Enhancement and Its Implementation on a Hearing Aid App.

Remixing Music with Visual Conditioning

Cooperative Audio Source Separation and Enhancement Using Distributed Microphone Arrays and Wearable Devices

Sub-band and Full-band Interactive U-Net with DPRNN for Demixing Cross-talk Stereo Music

A Deep-Learning Based Framework for Source Separation, Analysis, and Synthesis of Choral Ensembles

Sound Demixing Challenge 2023 Music Demixing Track Technical Report: TFC-TDF-UNet v3