Abstract:Sound propagation is the process by which sound energy travels through a medium, such as air, to the surrounding environment as sound waves. The room impulse response (RIR) describes this process and is influenced by the positions of the source and listener, the room's geometry, and its materials. Physics-based acoustic simulators have been used for decades to compute accurate RIRs for specific acoustic environments. However, we have encountered limitations with existing acoustic simulators. To address these limitations, we propose three novel solutions. First, we introduce a learning-based RIR generator that is two orders of magnitude faster than an interactive ray-tracing simulator. Our approach can be trained to input both statistical and traditional parameters directly, and it can generate both monaural and binaural RIRs for both reconstructed and synthetic 3D scenes. Our generated RIRs outperform interactive ray-tracing simulators in speech-processing applications, including ASR, Speech Enhancement, and Speech Separation. Secondly, we propose estimating RIRs from reverberant speech signals and visual cues without a 3D representation of the environment. By estimating RIRs from reverberant speech, we can augment training data to match test data, improving the word error rate of the ASR system. Our estimated RIRs achieve a 6.9% improvement over previous learning-based RIR estimators in far-field ASR tasks. We demonstrate that our audio-visual RIR estimator aids tasks like visual acoustic matching, novel-view acoustic synthesis, and voice dubbing, validated through perceptual evaluation. Finally, we introduce IR-GAN to augment accurate RIRs using real RIRs. IR-GAN parametrically controls acoustic parameters learned from real RIRs to generate new RIRs that imitate different acoustic environments, outperforming Ray-tracing simulators on the far-field ASR benchmark by 8.95%.

Simplification of Head-Related Impulse Response in Early Reflection Simulation

An Efficient Finite-Impulse-response Filter Model of Head-Related Impulse Response

Head-Related Transfer Function Modeling Based on Finite-Impulse Response

A computer simulation model for room sound field considering diffuse reflection

Hybrid Method of Image Source Model and Diffuse Surface Integration for the Simulation of Room Sound Field

Computer Simulation Model for Room Diffuse Sound Field

Dependency of the Finite-Impulse-Response-Based Head-Related Impulse Response Model on Filter Order

A Perceptual Approach for Interaural Transfer Function Calculations Based on Wiener Filtering

Binaural reproduction of dummy head and spherical microphone array data-A perceptual study on the minimum required spatial resolution

A Simplified Model for Generating 3D Realistic Sound in the Multimedia and Virtual Reality Systems

Time-of-arrival Estimation and Phase Unwrapping of Head-related Transfer Functions With Integer Linear Programming

Spatial resolution of late reverberation in virtual acoustic environments

Computationally-efficient and perceptually-motivated rendering of diffuse reflections in room acoustics simulation

Evaluation of Virtual Acoustic Environments with Different Acoustic Level of Detail

Synthesis of Room Responses Using the Virtual Source Representation and the Comb-Nested Allpass Filters

Interpolating the Directional Room Impulse Response for Dynamic Spatial Audio Reproduction

Numerical Modeling of Head-Related Transfer Functions Using the Virtual Source Representation

Modeling of reverberant room responses for two-dimensional spatial sound field analysis and synthesis.

A binaural room impulse response dataset and Shorelining psychophysical task for the evaluation of auditory sensory augmentation

Efficient learning-based sound propagation for virtual and real-world audio processing applications

Hearing Anything Anywhere