Abstract:Spatial audio has been studied for several decades, but has seen much renewed interest recently due to advances in both software and hardware for capture and playback, and the emergence of applications such as virtual reality and augmented reality. This renewed interest has led to the investment of increasing efforts in developing signal processing algorithms for spatial audio, both for capture and for playback. In particular, due to the popularity of headphones and earphones, many spatial audio signal processing methods have dealt with binaural reproduction based on headphone listening. Among these new developments, processing spatial audio signals recorded in real environments using microphone arrays plays an important role. Following this emerging activity, this paper aims to provide a scientific review of recent developments and an outlook for future challenges. This review also proposes a generalized framework for describing spatial audio signal processing for the binaural reproduction of recorded sound. This framework helps to understand the collective progress of the research community, and to identify gaps for future research. It is composed of five main blocks, namely: the acoustic scene, recording, processing, reproduction, and perception and evaluation. First, each block is briefly presented, and then, a comprehensive review of the processing block is provided. This includes topics from simple binaural recording to Ambisonics and perceptually motivated approaches, which focus on careful array configuration and design. Beamforming and parametric-based processing afford more flexible designs and shift the focus to processing and modeling of the sound field. Then, emerging machine- and deep-learning approaches, which take a further step towards flexibility in design, are described. Finally, specific methods for signal transformations such as rotation, translation and enhancement, enabling additional flexibility in reproduction and improvement in the quality of the binaural signal, are presented. The review concludes by highlighting directions for future research.

Separation of Overlapping Audio Signals: A Review on Current Trends and Evolving Approaches

Tackling the Cocktail Fork Problem for Separation and Transcription of Real-World Soundtracks

The Signal Separation Evaluation Campaign (2007-2010): Achievements and Remaining Challenges.

A Consolidated Perspective on Multimicrophone Speech Enhancement and Source Separation

Review of audio deepfake detection techniques: Issues and prospects

The Cocktail Fork Problem: Three-Stem Audio Separation for Real-World Soundtracks

Acoustic Source Localization and Deconvolution-Based Separation

Spatial audio signal processing for binaural reproduction of recorded acoustic scenes – review and challenges

Improving Sound Event Detection In Domestic Environments Using Sound Separation

MUSIC REMIXING AND UPMIXING USING SOURCE SEPARATION

Co-Separating Sounds of Visual Objects

Uncovering the Deceptions: An Analysis on Audio Spoofing Detection and Future Prospects

Should We Always Separate?: Switching Between Enhanced and Observed Signals for Overlapping Speech Recognition

OmniSep: Unified Omni-Modality Sound Separation with Query-Mixup

Improving Generalization of Speech Separation in Real-World Scenarios: Strategies in Simulation, Optimization, and Evaluation

Speech Enhancement Using Open-Unmix Music Source Separation Architecture

Separating Invisible Sounds Toward Universal Audiovisual Scene-Aware Sound Separation

Facing the Music: Tackling Singing Voice Separation in Cinematic Audio Source Separation

Sound Synthesis, Propagation, and Rendering: A Survey

AudioScopeV2: Audio-Visual Attention Architectures for Calibrated Open-Domain On-Screen Sound Separation

Continual Audio-Visual Sound Separation