Abstract:Binaural rendering is of great interest to virtual reality and immersive media. Although humans can naturally use their two ears to perceive the spatial information contained in sounds, it is a challenging task for machines to achieve binaural rendering since the description of a sound field often requires multiple channels and even the metadata of the sound sources. In addition, the perceived sound varies from person to person even in the same sound field. Previous methods generally rely on individual-dependent head-related transferred function (HRTF) datasets and optimization algorithms that act on HRTFs. In practical applications, there are two major drawbacks to existing methods. The first is a high personalization cost, as traditional methods achieve personalized needs by measuring HRTFs. The second is insufficient accuracy because the optimization goal of traditional methods is to retain another part of information that is more important in perception at the cost of discarding a part of the information. Therefore, it is desirable to develop novel techniques to achieve personalization and accuracy at a low cost. To this end, we focus on the binaural rendering of ambisonic and propose 1) channel-shared encoder and channel-compared attention integrated into neural networks and 2) a loss function quantifying interaural level differences to deal with spatial information. To verify the proposed method, we collect and release the first paired ambisonic-binaural dataset and introduce three metrics to evaluate the content information and spatial information accuracy of the end-to-end methods. Extensive experimental results on the collected dataset demonstrate the superior performance of the proposed method and the shortcomings of previous methods.

Low-complexity binaural decoding using time/frequency domain HRTF equalization

3D Audio Rendering in Distributed Virtual Environment

Robust Binaural Rendering with the Time-Domain Underdetermined Multichannel Inverse Prefilters.

Low Complexity Estimation and Equalization of Doubly Spread Underwater Acoustic Channels

Filtering Optimization in MDCT Domain Base on Spectrum Character of HRTF

A Simple Implementation for 3D Virtual Surround Sound Effect and Its Application in Multichannel Audio Coding

End-to-End Paired Ambisonic-Binaural Audio Rendering

Individualized HRTF-based Binaural Renderer for Higher-Order Ambisonics

Model-matching Principle Applied to the Design of an Array-based All-neural Binaural Rendering System for Audio Telepresence

Development and implementation of cross-talk cancellation system in spatial audio reproduction based on subband filtering

Binaural Rendering of Ambisonic Signals by Neural Networks

Comparative Study of Audio Spatializers for Dual-Loudspeaker Mobile Phones

A MVDR- MWF Combined Algorithm for Binaural Hearing Aid System

Interpolation Method of Head-Related Transfer Functions Based on Common-Pole/zero Modeling

Effective binaural multi-channel processing algorithm for improved environmental presence

Time-of-arrival Estimation and Phase Unwrapping of Head-related Transfer Functions With Integer Linear Programming

Hrtf Representation With Convolutional Auto-Encoder

HRTF Estimation in the Wild

iMagLS: Interaural Level Difference with Magnitude Least-Squares Loss for Optimized First-Order Head-Related Transfer Function

Low bit rate binaural link for improved ultra low-latency low-complexity multichannel speech enhancement in Hearing Aids

A refined algorithm of 3-D sound synthesis