Abstract:Three-dimensional sound's effectiveness in virtual reality (VR) environments has been widely studied. However, due to the big differences between VR and augmented reality (AR) systems in registration, calibration, perceptual difference of immersiveness, navigation, and localization, it is important to develop new approaches to seamlessly register virtual 3-D sound in AR environments and conduct studies on 3-D sound's effectiveness in AR context. In this paper, we design two experimental AR environments to study the effectiveness of 3-D sound both quantitatively and qualitatively. Two different tracking methods are applied to retrieve the 3-D position of virtual sound sources in each experiment. We examine the impacts of 3-D sound on improving depth perception and shortening task completion time. We also investigate its impacts on immersive and realistic perception, different spatial objects identification, and subjective feeling of "human presence and collaboration". Our studies show that applying 3-D sound is an effective way to complement visual AR environments. It helps depth perception and task performance, and facilitates collaborations between users. Moreover, it enables a more realistic environment and more immersive feeling of being inside the AR environment by both visual and auditory means. In order to make full use of the intensity cues provided by 3-D sound, a process to scale the intensity difference of 3-D sound at different depths is designed to cater small AR environments. The user study results show that the scaled 3-D sound significantly increases the accuracy of depth judgments and shortens the searching task completion time. This method provides a necessary foundation for implementing 3-D sound in small AR environments. Our user study results also show that this process does not degrade the intuitiveness and realism of an augmented audio reality environment

Visualization system for sound field using see-through head-mounted display

VisibleSound: Perceiving Environmental Sound with 4D Form

A Wearable Vision-To-Audio Sensory Substitution Device for Blind Assistance and the Correlated Neural Substrates

Multi-sensor Spatial Augmented Reality for Visualizing the Invisible Thermal Information of 3D Objects

Head-Mounted Display Visualizations to Support Sound Awareness for the Deaf and Hard of Hearing

Development of a Three-Dimensional Multimode Visual Immersive System With Applications in Telepresence

Video Visualization for Moving Sound Sources Based on Binoculars Stereo and Acoustical Holography

The Role of 3-D Sound in Human Reaction and Performance in Augmented Reality Environments

Simplified model for generating 3D realistic sound in the multimedia and virtual reality systems

A Multi-Plane Optical See-Through Head Mounted Display Design for Augmented Reality Applications

Visar: Projecting Virtual Sound Spots for Acoustic Augmented Reality Using Air Nonlinearity

3-1: A Multi-plane Volumetric Optical See-through Head Mounted 3D Display

A Multiplane Volumetric Optical See-through Head Mounted 3 D Display †

An Experimental Study on the Role of 3D Sound in Augmented Reality Environment.

3D Volume Visualization and Screen-based Interaction with Dynamic Ray Casting on Autostereoscopic Display

Head mounted device for gesture recognition using acoustic time of flight sensor and 2D camera

Physical-model-based reconstruction of three-dimensional sound field from multi-directional measurement by parallel phase-shift interferometry

Comparison of a Head-Mounted Display and a Curved Screen in a Multi-Talker Audiovisual Listening Task

Development of sound visualization system based on binocular vision and short-time wave superposition algorithm

Video Visualization For Moving Sound Sources Based On Binocular Vision And Short-Time Beamforming