Abstract:Interactive audio spatialization technology previously developed for video game authoring and rendering has evolved into an essential component of platforms enabling shared immersive virtual experiences for future co-presence, remote collaboration and entertainment applications. New wearable virtual and augmented reality displays employ real-time binaural audio computing engines rendering multiple digital objects and supporting the free navigation of networked participants or their avatars through a juxtaposition of environments, real and virtual, often referred to as the Metaverse. These applications require a parametric audio scene programming interface to facilitate the creation and deployment of shared, dynamic and realistic virtual 3D worlds on mobile computing platforms and remote servers. We propose a practical approach for designing parametric 6-degree-of-freedom object-based interactive audio engines to deliver the perceptually relevant binaural cues necessary for audio/visual and virtual/real congruence in Metaverse experiences. We address the effects of room reverberation, acoustic reflectors, and obstacles in both the virtual and real environments, and discuss how such effects may be driven by combinations of pre-computed and real-time acoustic propagation solvers. We envision an open scene description model distilled to facilitate the development of interoperable applications distributed across multiple platforms, where each audio object represents, to the user, a natural sound source having controllable distance, size, orientation, and acoustic radiation properties.

ARE: an Audio Reality Engine for Virtual Environments.

3D Audio Rendering in Distributed Virtual Environment

A real-time auralization framework for scientific experiments

An end-to-end approach for blindly rendering a virtual sound source in an audio augmented reality environment

A Simplified Model for Generating 3D Realistic Sound in the Multimedia and Virtual Reality Systems

Audio Rendering In Virtual Environment

Hearing Anything Anywhere

Room-aware portable Auditory Augmented Reality: Real-time spatial audio generation with geometric data analysis

Rendering Spatial Sound for Interoperable Experiences in the Audio Metaverse

Aural proxies and directionally-varying reverberation for interactive sound propagation in virtual environments

Simulation System to Construct Virtual Acoustic Environment Based on Dynamic Mechanisms

3D sound spatialization with game engines: the virtual acoustics performance of a game engine and a middleware for interactive audio design

Acoustic Simulation with Dynamic Mechanisms in Virtual Reality

Acoustic Volume Rendering for Neural Impulse Response Fields

WAVE: Interactive Wave-based Sound Propagation for Virtual Environments

Interpolating the Directional Room Impulse Response for Dynamic Spatial Audio Reproduction

Replicating outdoor environments using VR and ambisonics: a methodology for accurate audio-visual recording, processing and reproduction

Real-time auralization for performers on virtual stages

Auralization based on multi-perspective ambisonic room impulse responses

A Modified Radiosity Algorithm for Integrated Visual and Auditory Rendering

SOAF: Scene Occlusion-aware Neural Acoustic Field