Abstract:Auditory assessments aim to quantify auditory abilities that impact real-world listening, such as replicating patient complaints of listening difficulty in specific situations or understanding how auditory systems perform in natural scenes. This goal suggests that assessments should be as realistic as possible. However, the need to quantify specific abilities requires greater experimental control than achievable in most natural settings. This fundamental trade-off between realism and control has existed since perception science began. But emerging technologies in virtual and augmented reality promise to bring the real world into the lab–or vice versa–and break this trade-off. Implemented correctly, these technologies offer new approaches to quasi-realistic auditory assessment with improved multisensory consistency, more natural tasks, and greater participant engagement that will make assessment more reliable, valid, and fun. Traditional laboratory-based testing remains the gold standard for auditory assessment in the clinic and in basic research. But several features of this approach limit the real-world validity of laboratory tasks. First, these tasks are rife with multisensory conflict. Booth walls and desktop displays of the testing environment are visually distracting and inconsistent with the auditory stimuli listeners are asked to judge, whereas real-world listening involves a consistent multisensory context that can reduce uncertainty and support task performance even when non-auditory cues are not explicitly informative about the task. Removing that support increases demands on participants' memory and mental imagery—cognitive skills that can confound attempts to measure sensory abilities. Even in explicitly multisensory studies, the spatial arrangement of auditory and visual stimuli are often poorly matched: Visuals may appear on a desktop display, while sounds appear inside the head or from a distant loudspeaker. In contrast, real-world auditory and visual features are aligned in all directions and remain so as a person moves around, producing a natural experience of immersion within the sensory world. Second, laboratory-based tasks are behaviorally unrealistic, typically presenting explicit feedback about an isolated task dimension such as localization or identification. But real-world listening combines multiple task dimensions and implicit feedback. Conversation in a crowded restaurant, for example, requires simultaneous localization, segregation, and identification of target speech, along with the evaluation of facial cues that might indicate which talker asked an important question but not the content of the question itself. Virtual reality (VR) and other immersive technologies have been gaining attention as new modes for computer-based entertainment, gaming, communication, and work. VR head-mounted displays (HMDs) present video and audio content (typically a simulated 3D environment) customized for the individual user. Low-latency tracking of the user's position in space is used to update the display in close to real time, allowing direct interaction with immersive multisensory experiences. At Boys Town National Research Hospital's Spatial Hearing Laboratory, we have been using commercial HMDs to explore the benefits of VR for auditory assessment. The focus has been on enhancing auditory assessment rather than on assessing the performance of virtual experiences (e.g., 3D audio) themselves. As such, our approach has been to add immersive multisensory elements without giving up the control made possible by existing unisensory methods and apparatus. For computer-based assessment, this is accomplished by software commands sent to and from a dedicated process that controls VR-based interactions with the participant. Across multiple investigations, we have noted several exciting potential benefits: Improving Multisensory Consistency. Replacing the sound booth with even a simple virtual scene can eliminate distracting visual elements and provide better consistency between visual and auditory elements, which can help participants understand the auditory stimulus even when visual cues are not directly informative. Working in my lab at Vanderbilt University, Travis Moore, AuD, PhD, conducted a binaural localization study in which listeners indicated the direction of a perceived sound by turning their heads in a simple immersive scene—a circular array of 360 balloons (one per degree azimuth). On each trial, they turned to face the balloon closest to where they heard a target sound and pressed a button to pop the selected balloon with a simple silent animation. This VR-based approach (a) provided a more consistent and controlled visual background than the sound booth interior, (b) supported participants' sense of target sounds' spatial position relative to their own dynamic viewpoint, and (c) provided naturalistic feedback on th -Abstract Truncated-

Evaluating Spatial Hearing Using a Dual-Task Approach in a Virtual-Acoustics Environment

Spatial hearing training in virtual reality with simulated asymmetric hearing loss

Localization and Speech-in-Noise Performance in a Virtual Reality Test Environment: A Pilot Study of Adults With Single-Sided Deafness Using a Cochlear Implant

Spatial rehabilitation using virtual auditory space training paradigm in individuals with sensorineural hearing impairment

Auditory Spatial Discrimination and Sound Localization in Single-Sided Deaf Participants Provided with a Cochlear Implant

A Compact Two-Loudspeaker Virtual Sound Reproduction System for Clinical Testing of Spatial Hearing With Hearing-Assistive Devices

Evaluation of spatial audio reproduction schemes for application in hearing aid research

Active listening modulates the spatial hearing experience: a multicentric study

Auditory Spatial Bisection of Blind and Normally Sighted Individuals in Free Field and Virtual Acoustics

Toward Sound Localization Testing in Virtual Reality to Aid in the Screening of Auditory Processing Disorders

Development and evaluation of the LiSN & learn auditory training software for deficit-specific remediation of binaural processing deficits in children: preliminary findings

Spatial Hearing Difficulties in Reaching Space in Bilateral Cochlear Implant Children Improve With Head Movements

Evaluating on the Spatial Localization Cues of Binaural Sound Pressure Recorded Via Behind-the-ear Microphones in a Hearing Aid

Interleaved Acoustic Environments: Impact of an Auditory Scene Classification Procedure on Speech Perception in Cochlear Implant Users

Effectiveness of Directional Microphones in Bilateral/Bimodal Cochlear Implant Users-Impact of Spatial and Temporal Noise Characteristics

Validating a Method to Assess Lipreading, Audiovisual Gain, and Integration During Speech Reception With Cochlear-Implanted and Normal-Hearing Subjects Using a Talking Head

Sound Localization in Single-Sided Deaf Participants Provided With a Cochlear Implant

Evaluation of the Benefits of Bilateral Fitting in Bone-Anchored Hearing System Users: Spatial Resolution and Memory for Speech

Development and validation of a questionnaire for assessing visual and auditory spatial localization abilities in dual sensory impairment

Using Virtual Reality to Assess Auditory Performance

Spatial audio reproduction for studying second language speech perception in varying acoustic environments