Abstract:Auditory assessments aim to quantify auditory abilities that impact real-world listening, such as replicating patient complaints of listening difficulty in specific situations or understanding how auditory systems perform in natural scenes. This goal suggests that assessments should be as realistic as possible. However, the need to quantify specific abilities requires greater experimental control than achievable in most natural settings. This fundamental trade-off between realism and control has existed since perception science began. But emerging technologies in virtual and augmented reality promise to bring the real world into the lab–or vice versa–and break this trade-off. Implemented correctly, these technologies offer new approaches to quasi-realistic auditory assessment with improved multisensory consistency, more natural tasks, and greater participant engagement that will make assessment more reliable, valid, and fun. Traditional laboratory-based testing remains the gold standard for auditory assessment in the clinic and in basic research. But several features of this approach limit the real-world validity of laboratory tasks. First, these tasks are rife with multisensory conflict. Booth walls and desktop displays of the testing environment are visually distracting and inconsistent with the auditory stimuli listeners are asked to judge, whereas real-world listening involves a consistent multisensory context that can reduce uncertainty and support task performance even when non-auditory cues are not explicitly informative about the task. Removing that support increases demands on participants' memory and mental imagery—cognitive skills that can confound attempts to measure sensory abilities. Even in explicitly multisensory studies, the spatial arrangement of auditory and visual stimuli are often poorly matched: Visuals may appear on a desktop display, while sounds appear inside the head or from a distant loudspeaker. In contrast, real-world auditory and visual features are aligned in all directions and remain so as a person moves around, producing a natural experience of immersion within the sensory world. Second, laboratory-based tasks are behaviorally unrealistic, typically presenting explicit feedback about an isolated task dimension such as localization or identification. But real-world listening combines multiple task dimensions and implicit feedback. Conversation in a crowded restaurant, for example, requires simultaneous localization, segregation, and identification of target speech, along with the evaluation of facial cues that might indicate which talker asked an important question but not the content of the question itself. Virtual reality (VR) and other immersive technologies have been gaining attention as new modes for computer-based entertainment, gaming, communication, and work. VR head-mounted displays (HMDs) present video and audio content (typically a simulated 3D environment) customized for the individual user. Low-latency tracking of the user's position in space is used to update the display in close to real time, allowing direct interaction with immersive multisensory experiences. At Boys Town National Research Hospital's Spatial Hearing Laboratory, we have been using commercial HMDs to explore the benefits of VR for auditory assessment. The focus has been on enhancing auditory assessment rather than on assessing the performance of virtual experiences (e.g., 3D audio) themselves. As such, our approach has been to add immersive multisensory elements without giving up the control made possible by existing unisensory methods and apparatus. For computer-based assessment, this is accomplished by software commands sent to and from a dedicated process that controls VR-based interactions with the participant. Across multiple investigations, we have noted several exciting potential benefits: Improving Multisensory Consistency. Replacing the sound booth with even a simple virtual scene can eliminate distracting visual elements and provide better consistency between visual and auditory elements, which can help participants understand the auditory stimulus even when visual cues are not directly informative. Working in my lab at Vanderbilt University, Travis Moore, AuD, PhD, conducted a binaural localization study in which listeners indicated the direction of a perceived sound by turning their heads in a simple immersive scene—a circular array of 360 balloons (one per degree azimuth). On each trial, they turned to face the balloon closest to where they heard a target sound and pressed a button to pop the selected balloon with a simple silent animation. This VR-based approach (a) provided a more consistent and controlled visual background than the sound booth interior, (b) supported participants' sense of target sounds' spatial position relative to their own dynamic viewpoint, and (c) provided naturalistic feedback on th -Abstract Truncated-

The Effects of Signal to Noise Ratio, T 60, Wide-Dynamic Range Compression Speed, and Digital Noise Reduction in a Virtual Restaurant Setting

The Effects of Signal to Noise Ratio, T60 , Wide-Dynamic Range Compression Speed, and Digital Noise Reduction in a Virtual Restaurant Setting

Evaluation of Virtual Acoustic Environments with Different Acoustic Level of Detail

Effect of acoustic scene complexity and visual scene representation on auditory perception in virtual audio-visual environments

An ecological investigation of the effect of background noise on speech processing in a Virtual Classroom

Evaluating Spatial Hearing Using a Dual-Task Approach in a Virtual-Acoustics Environment

Evaluation of a Transient Noise Reduction Algorithm in Cochlear Implant Users

Modeling the effects of dynamic range compression on signals in noise

Efficacy and Effectiveness of Advanced Hearing Aid Directional and Noise Reduction Technologies for Older Adults With Mild to Moderate Hearing Loss

A Compact Two-Loudspeaker Virtual Sound Reproduction System for Clinical Testing of Spatial Hearing With Hearing-Assistive Devices

The Noise Reduction Algorithm May Not Compensate for the Degradation in Output Signal-to-Noise Ratio Caused by Wide Dynamic Range Compression

Using Virtual Reality to Systematically Examine Impacts of Noise and Visual Clutter on Message Elaboration and Cognitive Capacity

Benefit of Hearing-Aid Amplification and Signal Enhancement for Speech Reception in Complex Listening Situations

Using Virtual Reality to Assess Auditory Performance

Performance in noise: Impact of reduced speech intelligibility on Sailor performance in a Navy command and control environment

Simulating environmental and psychological acoustic factors of the operating room

The Impact of Hearing Aids on Listening Effort and Listening-Related Fatigue - Investigations in a Virtual Realistic Listening Environment

Localization and Speech-in-Noise Performance in a Virtual Reality Test Environment: A Pilot Study of Adults With Single-Sided Deafness Using a Cochlear Implant

Evaluating Real-World Benefits of Hearing Aids With Deep Neural Network–Based Noise Reduction: An Ecological Momentary Assessment Study

Relationship Between Working Memory, Compression, and Beamformers in Ideal Conditions

Effect of Hearing Device Use on Speech-in-Noise Performance in Children with Severe-to-Profound Unilateral Hearing Loss