Abstract:Assistive devices for visually impaired people (VIP) which support daily traveling and improve social inclusion are developing fast. Most of them try to solve the problem of navigation or obstacle avoidance, and other works focus on helping VIP to recognize their surrounding objects. However, very few of them couple both capabilities (i.e., navigation and recognition). Aiming at the above needs, this paper presents a wearable assistive device that allows VIP to (i) navigate safely and quickly in unfamiliar environment, and (ii) to recognize the objects in both indoor and outdoor environments. The device consists of a consumer Red, Green, Blue and Depth (RGB-D) camera and an Inertial Measurement Unit (IMU), which are mounted on a pair of eyeglasses, and a smartphone. The device leverages the ground height continuity among adjacent image frames to segment the ground accurately and rapidly, and then search the moving direction according to the ground. A lightweight Convolutional Neural Network (CNN)-based object recognition system is developed and deployed on the smartphone to increase the perception ability of VIP and promote the navigation system. It can provide the semantic information of surroundings, such as the categories, locations, and orientations of objects. Human–machine interaction is performed through audio module (a beeping sound for obstacle alert, speech recognition for understanding the user commands, and speech synthesis for expressing semantic information of surroundings). We evaluated the performance of the proposed system through many experiments conducted in both indoor and outdoor scenarios, demonstrating the efficiency and safety of the proposed assistive system.

Long-Range Traversability Awareness and Low-Lying Obstacle Negotiation with RealSense for the Visually Impaired

Expanding the Detection of Traversable Area with RealSense for the Visually Impaired

IR Stereo RealSense: Decreasing Minimum Range of Navigational Assistance for Visually Impaired Individuals.

Detecting Traversable Area and Water Hazards for the Visually Impaired with a Prgb-D Sensor.

Perception Framework of Water Hazards Beyond Traversability for Real-World Navigation Assistance Systems

Assisting the Visually Impaired: Multitarget Warning Through Millimeter Wave Radar and RGB-depth Sensors.

Glass Detection and Recognition Based on the Fusion of Ultrasonic Sensor and RGB-D Sensor for the Visually Impaired

Robustifying Semantic Cognition of Traversability Across Wearable RGB-depth Cameras

Unifying Terrain Awareness Through Real-Time Semantic Segmentation

Improving RealSense by Fusing Color Stereo Vision and Infrared Stereo Vision for the Visually Impaired

Intersection Navigation For People With Visual Impairment

A New Approach of Point Cloud Processing and Scene Segmentation for Guiding the Visually Impaired

Semantic perception of curbs beyond traversability for real-world navigation assistance systems

An Environmental Perception and Navigational Assistance System for Visually Impaired Persons Based on Semantic Stixels and Sound Interaction

Reducing the minimum range of a RGB-depth sensor to aid navigation in visually impaired individuals

Unifying Terrain Awareness for the Visually Impaired through Real-Time Semantic Segmentation

Wearable Travel Aid for Environment Perception and Navigation of Visually Impaired People

Fusion of millimeter wave radar and RGB-depth sensors for assisted navigation of the visually impaired

Indoor Navigation Assistance for Visually Impaired People via Dynamic SLAM and Panoptic Segmentation with an RGB-D Sensor

HIDA: Towards Holistic Indoor Understanding for the Visually Impaired via Semantic Instance Segmentation with a Wearable Solid-State LiDAR Sensor

A Wearable Navigation Device for Visually Impaired People Based on the Real-Time Semantic Visual SLAM System