Abstract:Background and objectives: In order to improve assistive technologies for people with reduced mobility, this paper develops a new intelligent real-time emotion detection system to control equipment, such as electric wheelchairs (EWC) or robotic assistance vehicles. Every year, degenerative diseases and traumas prohibit thousands of people to easily control the joystick of their wheelchairs with their hands. Most current technologies are considered invasive and uncomfortable such as those requiring the user to wear some body sensor to control the wheelchair. Methods: In this work, the proposed Human Machine Interface (HMI) provides an efficient hands-free option that does not require sensors or objects attached to the user's body. It allows the user to drive the wheelchair using its facial expressions which can be flexibly updated. This intelligent solution is based on a combination of neural networks (NN) and specific image preprocessing steps. First, the Viola-Jones combination is used to detect the face of the disability from a video. Subsequently, a neural network is used to classify the emotions displayed on the face. This solution called "The Mathematics Behind Emotion" is capable of classifying many facial expressions in real time, such as smiles and raised eyebrows, which are translated into signals for wheelchair control. On the hardware side, this solution only requires a smartphone and a Raspberry Pi card that can be easily mounted on the wheelchair. Results: Many experiments have been conducted to evaluate the efficiency of the control acquisition process and the user experience in driving a wheelchair through facial expressions. The classification accuracy can expect 98.6% and it can offer an average recall rate of 97.1%. Thus, all these experiments have proven that the proposed system is able of accurately recognizing user commands in real time. Indeed, the obtained results indicate that the suggested system is more comfortable and better adapted to severely disabled people in their daily lives, than conventional methods. Among the advantages of this system, we cite its real time ability to identify facial emotions from different angles. Conclusions: The proposed system takes into account the patient's pathology. It is intuitive, modern, doesn't require physical effort and can be integrated into a smartphone or tablet. The results obtained highlight the efficiency and reliability of this system, which ensures safe navigation for the disabled patient.

Conveying facial expressions to blind and visually impaired persons through a wearable vibrotactile device

Functional performance of a vibrotactile sensory substitution device in people with profound vision loss

A Wearable Vision-To-Audio Sensory Substitution Device for Blind Assistance and the Correlated Neural Substrates

An Evaluation of a Wearable Assistive Device for Augmenting Social Interactions.

EmoSparkle: Tangible Prototype to Convey Visual Expressions for Visually Impaired Individuals in Real-time Conversations.

A Virtual Vision Navigation System for The Blind Using Wearable Touch-vision Devices

An investigation into the effectiveness of using acoustic touch to assist people who are blind

A Vibro-tactile Stimulation and Vibro-signature Synchronization Device for SSSEP-Based Study.

Characterization, Experimental Validation and Pilot User Study of the Vibro-Inertial Bionic Enhancement System (VIBES)

Wearable Visually Assistive Device for Blind People to Appreciate Real-world Scene and Screen Image

A facial expression controlled wheelchair for people with disabilities

Design Of A Vibrotactile Vest For Contour Perception

VIBES: Vibro-Inertial Bionic Enhancement System in a Prosthetic Socket

Remote Virtual Companion Via Tactile Codes and Voices for the People with Visual Impairment

FaceLooks: A Smart Headband for Signaling Face-to-Face Behavior

Electromagnetic actuation for a vibrotactile display: Assessing stimuli complexity and usability

A Multimodal Assistive System for Helping Visually Impaired in Social Interactions

Wearable Travel Aid for Environment Perception and Navigation of Visually Impaired People

A Multisensory Approach to Present Phonemes as Language Through a Wearable Haptic Device

Understanding Visually Impaired People's Experiences of Social Signal Perception in Face-to-face Communication.

Cross-Modal Sensory Boosting to Improve High-Frequency Hearing Loss: Device Development and Validation