Traffic light and moving object detection for a guide‐dog robot

Qiang Chen,Yinong Chen,Jinhui Zhu,Gennaro De Luca,Mei Zhang,Ying Guo

DOI: https://doi.org/10.1049/joe.2019.1137

2020-07-01

The Journal of Engineering

Abstract:Guide dogs are helpful for visually impaired people for navigating through the streets. However, it is expensive and time consuming to train a guide dog. In addition, a guide dog cannot decide when and where to cross a street safely, and it is up to the human to decide. Here, the authors propose a framework for creating a guide dog robot by using artificial intelligence and other technologies. The proposed framework is based on an Intel UP squared board, together with a Neural Compute Stick Movidius to process the images gathered from a GoPro camera. MobileNet single shot detector (SSD) is the main framework to detect the moving objects in the environment. The final decision is made after fusing the information gathered from all the sources. The authors also apply the Amazon Alexa device for the voice communication between the guide dog robot and the visually impaired person. A prototype of the proposed system is implemented and tested. Experimental results show that the proposed framework can process the information at a traffic intersection scene and can guide a blind person to cross the street safely.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to provide a safer and more effective street - crossing assistance tool for the visually impaired. Specifically, the author proposes a guide - dog - robot framework based on artificial intelligence and other modern technologies to address the problems of high training cost, long training time of traditional guide dogs, and their limited ability in determining when and where it is safe to cross the street. This framework helps the visually impaired cross the street safely by using visual perception technology to identify traffic lights, detect moving objects, and measure the distance between the visually impaired and obstacles.

The main problems mentioned in the paper include:
1. **Limitations of traditional guide dogs**: High training cost and long training time; unable to independently judge when and where it is safe to cross the street and need to rely on human decision - making.
2. **Deficiencies of existing technologies**: For example, although the system proposed by Chuang et al. can work effectively in an indoor environment, it uses the CaffeNet network with high computational cost, and mainly relies on image input to decide turning, lacking the detection of traffic lights and surrounding moving objects, resulting in the visually impaired being unable to fully perceive the environment.

To solve these problems, the author proposes a new guide - dog - robot framework based on service - computing architecture, Internet - of - Things technology, and the concept of robot - as - a - service. This framework utilizes low - cost hardware platforms (such as Intel UP squared board and Intel Movidius neural - computing stick) and lightweight deep - learning models (such as MobileNet - SSD) to achieve efficient traffic - light recognition and moving - object detection, and can provide real - time advice on safe street - crossing to the visually impaired.