RG4Face: A serious game incorporating mime movement recognition for facial rehabilitation

E. Fontenele, José,da Silva Souza, Herika,Moreira, Rayele
DOI: https://doi.org/10.1007/s11042-024-20429-y
IF: 2.577
2024-11-08
Multimedia Tools and Applications
Abstract:The human face provides clues about identity, age, race, affect, health and communication. Also, facial expression plays an important role in non-verbal social interactions. Impairment of the facial muscles' motor part significantly impacts the patient's quality of life. One of the challenges in motor rehabilitation is keeping the patient's engagement and motivation in performing exercises. This work presents and evaluates a serious game, named RG4Face , developed to aid in the facial rehabilitation of patients with facial paralysis. RG4Face includes the recognition of six movements used in mime therapy to improve the strength and mobility of the facial muscles. Movement recognition was implemented according to the distance values between landmarks, obtained from an AI-based human face mesh estimation model. To evaluate the game, three experiments were carried out: (1) usability and User eXperience (UX), (2) performance evaluation of the recognition techniques of mime movements, and (3) patient perception evaluation. Regarding the usability and UX experiment, all participants positively evaluated the system in the System Usability Scale (score=91.28), and the User Experience Questionnaire scales all fell within the highest range (scores between 2.30 and 2.73). In the movement recognition experiment, techniques demonstrated high performance with an F1-Score ranging from 0.89 to 0.97 across the six face movements. Also, the results showed that patients had a positive perception. The experiments demonstrated the potential of RG4Face as a tool for facial rehabilitation. We conclude that the proposed game is ready for clinical trials with patients.
computer science, information systems, theory & methods,engineering, electrical & electronic, software engineering
What problem does this paper attempt to address?