Abstract:Vision-based localization approaches now underpin newly emerging navigation pipelines for myriad use cases from robotics to assistive technologies. Compared to sensor-based solutions, vision-based localization does not require pre-installed sensor infrastructure, which is costly, time-consuming, and/or often infeasible at scale. Herein, we propose a novel vision-based localization pipeline for a specific use case: navigation support for end-users with blindness and low vision. Given a query image taken by an end-user on a mobile application, the pipeline leverages a visual place recognition (VPR) algorithm to find similar images in a reference image database of the target space. The geolocations of these similar images are utilized in downstream tasks that employ a weighted-average method to estimate the end-user's location and a perspective-n-point (PnP) algorithm to estimate the end-user's direction. Additionally, this system implements Dijkstra's algorithm to calculate a shortest path based on a navigable map that includes trip origin and destination. The topometric map used for localization and navigation is built using a customized graphical user interface that projects a 3D reconstructed sparse map, built from a sequence of images, to the corresponding a priori 2D floor plan. Sequential images used for map construction can be collected in a pre-mapping step or scavenged through public databases/citizen science. The end-to-end system can be installed on any internet-accessible device with a camera that hosts a custom mobile application. For evaluation purposes, mapping and localization were tested in a complex hospital environment. The evaluation results demonstrate that our system can achieve localization with an average error of less than 1 meter without knowledge of the camera's intrinsic parameters, such as focal length.

ARMSAINTS: an AR-based Real-time Mobile System for Assistive Indoor Navigation with Target Segmentation

Long-Range Traversability Awareness and Low-Lying Obstacle Negotiation with RealSense for the Visually Impaired

Unifying Terrain Awareness Through Real-Time Semantic Segmentation

Expanding the Detection of Traversable Area with RealSense for the Visually Impaired

Vision-Based Mobile Indoor Assistive Navigation Aid for Blind People

Towards Semi-automatic Detection and Localization of Indoor Accessibility Issues using Mobile Depth Scanning and Computer Vision

Indoor Navigation Assistance for Visually Impaired People via Dynamic SLAM and Panoptic Segmentation with an RGB-D Sensor

A Visual-Inertial Fusion Based Tracking System For Mobile Augmented Reality

HIDA: Towards Holistic Indoor Understanding for the Visually Impaired via Semantic Instance Segmentation with a Wearable Solid-State LiDAR Sensor

Can We Unify Perception and Localization in Assisted Navigation? An Indoor Semantic Visual Positioning System for Visually Impaired People

Detect and Approach: Close-Range Navigation Support for People with Blindness and Low Vision

Visual Indoor Navigation Using Mobile Augmented Reality

Localization and Navigation System for Indoor Mobile Robot

Real-time Indoor Assistive Localization with Mobile Omnidirectional Vision and Cloud GPU Acceleration

Unifying Terrain Awareness for the Visually Impaired through Real-Time Semantic Segmentation

Outdoor Navigation Assistive System Based on Robust and Real-Time Visual-Auditory Substitution Approach

Real and virtual explorations of the environment and interactive tracking of movable objects for the blind on the basis of tactile-acoustical maps and 3D environment models

A Large-Scale Indoor Layout Reconstruction and Localization System for Spatial-Aware Mobile AR Applications

Sensing and Navigation of Wearable Assistance Cognitive Systems for the Visually Impaired

Autonomous Mapping and Navigation using Fiducial Markers and Pan-Tilt Camera for Assisting Indoor Mobility of Blind and Visually Impaired People

UNav: An Infrastructure-Independent Vision-Based Navigation System for People with Blindness and Low vision