Using Virtual Reality to Assess Auditory Performance
G. Christopher Stecker
DOI: https://doi.org/10.1097/01.HJ.0000558464.75151.52
2019-10-07
Hearing Journal
Abstract:Auditory assessments aim to quantify auditory abilities that impact real-world listening, such as replicating patient complaints of listening difficulty in specific situations or understanding how auditory systems perform in natural scenes. This goal suggests that assessments should be as realistic as possible. However, the need to quantify specific abilities requires greater experimental control than achievable in most natural settings. This fundamental trade-off between realism and control has existed since perception science began. But emerging technologies in virtual and augmented reality promise to bring the real world into the lab–or vice versa–and break this trade-off. Implemented correctly, these technologies offer new approaches to quasi-realistic auditory assessment with improved multisensory consistency, more natural tasks, and greater participant engagement that will make assessment more reliable, valid, and fun. Traditional laboratory-based testing remains the gold standard for auditory assessment in the clinic and in basic research. But several features of this approach limit the real-world validity of laboratory tasks. First, these tasks are rife with multisensory conflict. Booth walls and desktop displays of the testing environment are visually distracting and inconsistent with the auditory stimuli listeners are asked to judge, whereas real-world listening involves a consistent multisensory context that can reduce uncertainty and support task performance even when non-auditory cues are not explicitly informative about the task. Removing that support increases demands on participants' memory and mental imagery—cognitive skills that can confound attempts to measure sensory abilities. Even in explicitly multisensory studies, the spatial arrangement of auditory and visual stimuli are often poorly matched: Visuals may appear on a desktop display, while sounds appear inside the head or from a distant loudspeaker. In contrast, real-world auditory and visual features are aligned in all directions and remain so as a person moves around, producing a natural experience of immersion within the sensory world. Second, laboratory-based tasks are behaviorally unrealistic, typically presenting explicit feedback about an isolated task dimension such as localization or identification. But real-world listening combines multiple task dimensions and implicit feedback. Conversation in a crowded restaurant, for example, requires simultaneous localization, segregation, and identification of target speech, along with the evaluation of facial cues that might indicate which talker asked an important question but not the content of the question itself. Virtual reality (VR) and other immersive technologies have been gaining attention as new modes for computer-based entertainment, gaming, communication, and work. VR head-mounted displays (HMDs) present video and audio content (typically a simulated 3D environment) customized for the individual user. Low-latency tracking of the user's position in space is used to update the display in close to real time, allowing direct interaction with immersive multisensory experiences. At Boys Town National Research Hospital's Spatial Hearing Laboratory, we have been using commercial HMDs to explore the benefits of VR for auditory assessment. The focus has been on enhancing auditory assessment rather than on assessing the performance of virtual experiences (e.g., 3D audio) themselves. As such, our approach has been to add immersive multisensory elements without giving up the control made possible by existing unisensory methods and apparatus. For computer-based assessment, this is accomplished by software commands sent to and from a dedicated process that controls VR-based interactions with the participant. Across multiple investigations, we have noted several exciting potential benefits: Improving Multisensory Consistency. Replacing the sound booth with even a simple virtual scene can eliminate distracting visual elements and provide better consistency between visual and auditory elements, which can help participants understand the auditory stimulus even when visual cues are not directly informative. Working in my lab at Vanderbilt University, Travis Moore, AuD, PhD, conducted a binaural localization study in which listeners indicated the direction of a perceived sound by turning their heads in a simple immersive scene—a circular array of 360 balloons (one per degree azimuth). On each trial, they turned to face the balloon closest to where they heard a target sound and pressed a button to pop the selected balloon with a simple silent animation. This VR-based approach (a) provided a more consistent and controlled visual background than the sound booth interior, (b) supported participants' sense of target sounds' spatial position relative to their own dynamic viewpoint, and (c) provided naturalistic feedback on th -Abstract Truncated-
English Else