SightAid: empowering the visually impaired in the Kingdom of Saudi Arabia (KSA) with deep learning-based intelligent wearable vision system
Fatma M. Talaat,Mohammed Farsi,Mahmoud Badawy,Mostafa Elhosseini
DOI: https://doi.org/10.1007/s00521-024-09619-9
2024-03-29
Neural Computing and Applications
Abstract:In the Kingdom of Saudi Arabia, visual impairment poses significant challenges for approximately 17.5% of school-aged children, mainly due to refractive errors. These challenges extend to everyday navigation, environmental interaction, and overall life quality. Motivated by the desire to empower visually impaired individuals, who face navigational limitations, difficulties in object recognition, and inadequate assistance from traditional technologies, we propose SightAid. This innovative wearable vision system utilizes a deep learning-based framework, addressing the gaps left by current assistive solutions. Traditional methods, such as canes and GPS devices, often fail to meet the nuanced and dynamic needs of the visually impaired, especially in accurately identifying objects, understanding complex environments, and providing essential real-time feedback for independent navigation. SightAid comprises a seven-phase framework involving data collection, preprocessing, and training of a sophisticated deep neural network with multiple convolutional and fully connected layers. This system is integrated into smart glasses with augmented reality displays, enabling real-time object detection and recognition. Interaction with users is facilitated through audio or haptic feedback, informing them about the location and type of objects detected. A continuous learning mechanism, incorporating user feedback and new data, ensures the system's ongoing refinement and adaptability. For performance assessment, we utilized the MNIST dataset, and an Indoor Objects Detection dataset tailored for the visually impaired, featuring images of everyday objects crucial for safe indoor navigation. SightAid demonstrates remarkable performance with accuracy up to 0.9874, recall values between 0.98 and 0.99, F1-scores ranging from 0.98 to 0.99, and AUC-ROC values reaching as high as 0.9999. These metrics significantly surpass those of traditional methods, highlighting SightAid's potential to substantially improve the independence and safety of visually impaired individuals in various environments.
computer science, artificial intelligence