Abstract:Binaural rendering is of great interest to virtual reality and immersive media. Although humans can naturally use their two ears to perceive the spatial information contained in sounds, it is a challenging task for machines to achieve binaural rendering since the description of a sound field often requires multiple channels and even the metadata of the sound sources. In addition, the perceived sound varies from person to person even in the same sound field. Previous methods generally rely on individual-dependent head-related transferred function (HRTF) datasets and optimization algorithms that act on HRTFs. In practical applications, there are two major drawbacks to existing methods. The first is a high personalization cost, as traditional methods achieve personalized needs by measuring HRTFs. The second is insufficient accuracy because the optimization goal of traditional methods is to retain another part of information that is more important in perception at the cost of discarding a part of the information. Therefore, it is desirable to develop novel techniques to achieve personalization and accuracy at a low cost. To this end, we focus on the binaural rendering of ambisonic and propose 1) channel-shared encoder and channel-compared attention integrated into neural networks and 2) a loss function quantifying interaural level differences to deal with spatial information. To verify the proposed method, we collect and release the first paired ambisonic-binaural dataset and introduce three metrics to evaluate the content information and spatial information accuracy of the end-to-end methods. Extensive experimental results on the collected dataset demonstrate the superior performance of the proposed method and the shortcomings of previous methods.

A Simplified Model for Generating 3D Realistic Sound in the Multimedia and Virtual Reality Systems

3D Audio Rendering in Distributed Virtual Environment

Real-Time Space 3D Acoustic Location Based on Monte Carlo Algorithm

Sound Field Translation and Mixed Source Model for Virtual Applications with Perceptual Validation

Simulation System to Construct Virtual Acoustic Environment Based on Dynamic Mechanisms

Modeling and Driving Human Body Soundfields through Acoustic Primitives

Hearing Anything Anywhere

A Simple Implementation for 3D Virtual Surround Sound Effect and Its Application in Multichannel Audio Coding

Sounding Bodies: Modeling 3D Spatial Sound of Humans Using Body Pose and Audio

Acoustic Simulation with Dynamic Mechanisms in Virtual Reality

A refined algorithm of 3-D sound synthesis

Sound Synthesis, Propagation, and Rendering: A Survey

Sound localization in web-based 3D environments

Points2Sound: From mono to binaural audio using 3D point cloud scenes

A multi-attribute subjective evaluation method on binaural 3D audio without reference stimulus

Time-space analysis, synthesis, and room response modeling of virtual-reality sound fields using microphone and loudspeaker arrays

End-to-End Paired Ambisonic-Binaural Audio Rendering

Three-Dimensional Sound Field Reconstruction and Sound Power Estimation by Stereo Vision and Beamforming Technology

A real-time 3D sound localization system with miniature microphone array for virtual reality

Modeling of reverberant room responses for two-dimensional spatial sound field analysis and synthesis.

Analysis and synthesis of three-dimensional sound fields using a hybrid microphone array and a 3-layered loudspeaker array