Abstract:Objective: The human auditory system acquires environmental information under sound stimuli faster than visual or touch systems, which in turn, allows for faster human responses to such stimuli. It also complements senses such as sight, where direct line-of-view is necessary to identify objects, in the environment recognition process. This work focuses on implementing human reaction to sound stimuli and environment recognition on assistive robotic devices, such as robotic wheelchairs or robotized cars. These vehicles need environment information to ensure safe navigation. Approach: In the field of environment recognition, range sensors (such as LiDAR and ultrasonic systems) and artificial vision devices are widely used; however, these sensors depend on environment constraints (such as lighting variability or color of objects), and sound can provide important information for the characterization of an environment. In this work, we propose a sound-based approach to enhance the environment recognition process, mainly for cases that compromise human integrity, according to the International Classification of Functioning (ICF). Our proposal is based on a neural network implementation that is able to classify up to 15 different environments, each selected according to the ICF considerations on environment factors in the community-based physical activities of people with disabilities. Main results: The accuracy rates in environment classification ranges from 84% to 93%. This classification is later used to constrain assistive vehicle navigation in order to protect the user during daily activities. This work also includes real-time outdoor experimentation (performed on an assistive vehicle) by seven volunteers with different disabilities (but without cognitive impairment and experienced in the use of wheelchairs), statistical validation, comparison with previously published work, and a discussion section where the pros and cons of our system are evaluated. Significance: The proposed sound-based system is very efficient at providing general descriptions of the environment. Such descriptions are focused on vulnerable situations described by the ICF. The volunteers answered a questionnaire regarding the importance of constraining the vehicle velocities in risky environments, showing that all the volunteers felt comfortable with the system and its performance.

Environmental Sounds as Concept Carriers for Communication

SoundNet: investigating a language composed of environmental sounds

ENVIRONMENTAL SOUNDS AS CONCEPT CARRIERS FOR COMMUNICATION

VisibleSound: Perceiving Environmental Sound with 4D Form

A Comparative Study of Sonification Methods to Represent Distance and Forward-Direction in Pedestrian Navigation

A Perceptual Study of Sound Ecology in Peripheral Sonification

Sonification Aesthetics and Listening for Network Situational Awareness

Data sonification: do you see what I hear?

SoundSignaling: Realtime, Stylistic Modification of a Personal Music Corpus for Information Delivery

An Exploratory Study on Proposed New Sounds for Future Products

Spatial Sound Design in a Car Cockpit: Challenges and Perspectives

SonifyAR: Context-Aware Sound Generation in Augmented Reality

From 3D landscape visualization to environmental simulation: The contribution of sound to the perception of virtual environments

SonifEye: Sonification of Visual Information Using Physical Modeling Sound Synthesis

Interactive Sonification for Data Exploration: How listening modes and display purposes define design guidelines

Sound Space: Communicating Acoustics Through Interactive Visualization

How Does Visual Context of Urban Open Spaces Drive Soundscape Pleasantness and Eventfulness: a Laboratory Experiment Using a Higher Order Ambisonics Speaker System and Immersive Virtual Reality

Sounds of Green: How Brand Name Sounds Metaphorically Convey Environmental Friendliness

Towards better visualizations of urban sound environments: insights from interviews

Semantic labeling of nonspeech audio clips

Human-inspired sound environment recognition system for assistive vehicles