Abstract:Binaural rendering is of great interest to virtual reality and immersive media. Although humans can naturally use their two ears to perceive the spatial information contained in sounds, it is a challenging task for machines to achieve binaural rendering since the description of a sound field often requires multiple channels and even the metadata of the sound sources. In addition, the perceived sound varies from person to person even in the same sound field. Previous methods generally rely on individual-dependent head-related transferred function (HRTF) datasets and optimization algorithms that act on HRTFs. In practical applications, there are two major drawbacks to existing methods. The first is a high personalization cost, as traditional methods achieve personalized needs by measuring HRTFs. The second is insufficient accuracy because the optimization goal of traditional methods is to retain another part of information that is more important in perception at the cost of discarding a part of the information. Therefore, it is desirable to develop novel techniques to achieve personalization and accuracy at a low cost. To this end, we focus on the binaural rendering of ambisonic and propose 1) channel-shared encoder and channel-compared attention integrated into neural networks and 2) a loss function quantifying interaural level differences to deal with spatial information. To verify the proposed method, we collect and release the first paired ambisonic-binaural dataset and introduce three metrics to evaluate the content information and spatial information accuracy of the end-to-end methods. Extensive experimental results on the collected dataset demonstrate the superior performance of the proposed method and the shortcomings of previous methods.

A Source Separation Approach for the Combined SBA Signals in the Joint Representation of OBA and SBA

A New Seperation Method for Rotationg and Static Sound Source Localization

Direction Specific Ambisonics Source Separation with End-To-End Deep Learning

Sep-Stereo: Visually Guided Stereophonic Audio Generation by Associating Source Separation

SpaIn-Net: Spatially-Informed Stereophonic Music Source Separation

Leveraging Moving Sound Source Trajectories for Universal Sound Separation

Delay-and-Sum Beamforming Based Spatial Mapping for Multi-Source Sound Localization

Video-Guided Sound Source Separation

Diff-SAGe: End-to-End Spatial Audio Generation Using Diffusion Models

Multichannel audio signal source separation based on an Interchannel Loudness Vector Sum

Visually Guided Sound Source Separation Using Cascaded Opponent Filter Network

Binaural Scene Analysis with Multidimensional Statistical Filters

End-to-End Paired Ambisonic-Binaural Audio Rendering

Joint DOA Estimation and Dereverberation Based on Multi-Channel Linear Prediction Filtering and Azimuth Sparsity

SOAF: Scene Occlusion-aware Neural Acoustic Field

Frequency Domain Singular Value Decomposition for Efficient Spatial Audio Coding

Spatial Sound Field Synthesis and Upmixing Based on the Equivalent Source Method.

LAVSS: Location-Guided Audio-Visual Spatial Audio Separation

Self-supervised Learning of Audio Representations from Audio-Visual Data using Spatial Alignment

Analysis and synthesis of spatial sound fields with room response modeling using an integrated microphone and loudspeaker array system

Quasi-Blind Source Separation Algorithm for Convolutive Mixture of Speech