Abstract:Assistive technologies aim at enhancing personal mobility of individuals with disabilities to improve their independence and access to social life. For the visually impaired, perception during navigation comprises a major ingredient of independent living. With the development of computer vision, it is possible to meet the richer needs of visually impaired people. However, research on navigation assistance for the visually impaired is still relatively unexplored when compared with the active progress in autonomous driving which is already in full swing. In respond to this issue, we aim to leverage the study of the Stixel-World for automotive systems and transfer it to develop assistive technology for visually impaired people. The impressive research results of deep learning also suppose benefits for vision-based technology. Precisely, semantic segmentation is a task that enables identification of different objects uniformly. Inspired by these observations, we design a set of wearable visual aids, while the core algorithm is based on the stixel representations for three-dimensional world combined with pixel-wise semantic segmentation. Predetermined conditions for stixels in automotive research, such as camera angles, position fixes, and unsuitable assumptions made about the real world are optimized to fit the needs of navigation assistance in our algorithm, along with the incorporation of traversability-related semantic information. We also propose a sound mapping scheme, so that the environmental awareness about geographic and semantic information are conveyed to the visually impaired through acoustic feedback.

Research on the Peripheral Sound Visualization Using the Improved Ripple Mode

3D Audio Rendering in Distributed Virtual Environment

A Wearable Vision-To-Audio Sensory Substitution Device for Blind Assistance and the Correlated Neural Substrates

An Environmental Perception and Navigational Assistance System for Visually Impaired Persons Based on Semantic Stixels and Sound Interaction

Acoustic Field Visualization and Source Localization Via Physics-Informed Learning of Sparse Data with Adaptive Sampling

Poster: Inaudible High-throughput Communication Through Acoustic Signals

VisibleSound: Perceiving Environmental Sound with 4D Form

Perception-Aware Cross-Modal Signal Reconstruction: From Audio-Haptic to Visual

Head-Mounted Display Visualizations to Support Sound Awareness for the Deaf and Hard of Hearing

Video Visualization For Moving Sound Sources Based On Binocular Vision And Short-Time Beamforming

Development of sound visualization system based on binocular vision and short-time wave superposition algorithm

A Comparative Study in Real-Time Scene Sonification for Visually Impaired People

A Simplified Model for Generating 3D Realistic Sound in the Multimedia and Virtual Reality Systems

Perceiving Graphical and Pictorial Information via Hearing and Touch

Visual Sound Localization in the Wild by Cross-Modal Interference Erasing

A Basic Psychophysics Study of Sound Reliability Effects on Audiovisual Integration for Developing New Virtual Reality Device

Auditory Noise Leads to Increased Visual Brain-Computer Interface Performance: A Cross-Modal Study

A binaural room impulse response dataset and Shorelining psychophysical task for the evaluation of auditory sensory augmentation

Automatic Speech Recognition and its Visual Perception Via a Cymatics Based Display

Study on Artificial Imaging at The Peripheral Retina Region

Enhancing Situational Awareness with VAS-Compass Net for the Recognition of Directional Vehicle Alert Sounds