Can we cover navigational perception needs of the visually impaired by panoptic segmentation?

Wei Mao,Jiaming Zhang,Kailun Yang,Rainer Stiefelhagen

DOI: https://doi.org/10.48550/arXiv.2007.10202

2020-07-20

Abstract:Navigational perception for visually impaired people has been substantially promoted by both classic and deep learning based segmentation methods. In classic visual recognition methods, the segmentation models are mostly object-dependent, which means a specific algorithm has to be devised for the object of interest. In contrast, deep learning based models such as instance segmentation and semantic segmentation allow to individually recognize part of the entire scene, namely things or stuff, for blind individuals. However, both of them can not provide a holistic understanding of the surroundings for the visually impaired. Panoptic segmentation is a newly proposed visual model with the aim of unifying semantic segmentation and instance segmentation. Motivated by that, we propose to utilize panoptic segmentation as an approach to navigating visually impaired people by offering both things and stuff awareness in the proximity of the visually impaired. We demonstrate that panoptic segmentation is able to equip the visually impaired with a holistic real-world scene perception through a wearable assistive system.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to meet the navigation perception needs of the visually impaired through panoptic segmentation technology. Specifically, the paper explores how to use the panoptic segmentation method to simultaneously provide the identification of things and stuff in the surrounding environment, so as to help the visually impaired obtain an overall understanding of the surrounding environment. Traditional methods such as instance segmentation and semantic segmentation can only provide an understanding of part of the scene respectively, and cannot provide comprehensive information. By combining the advantages of these two methods, panoptic segmentation can provide a more complete and comprehensive scene perception solution for the visually impaired, thereby enhancing their independent navigation ability.