A Novel Speech Analysis and Correction Tool for Arabic-Speaking Children

Lamia Berriche,Maha Driss,Areej Ahmed Almuntashri,Asma Mufreh Lghabi,Heba Saleh Almudhi,Munerah Abdul-Aziz Almansour
2024-11-18
Abstract:This paper introduces a new application named ArPA for Arabic kids who have trouble with pronunciation. Our application comprises two key components: the diagnostic module and the therapeutic module. The diagnostic process involves capturing the child's speech signal, preprocessing, and analyzing it using different machine learning classifiers like K-Nearest Neighbors (KNN), Support Vector Machine (SVM), and Decision Trees as well as deep neural network classifiers like ResNet18. The therapeutic module offers eye-catching gamified interfaces in which each correctly spoken letter earns a higher avatar level, providing positive reinforcement for the child's pronunciation improvement. Two datasets were used for experimental evaluation: one from a childcare centre and the other including Arabic alphabet pronunciation recordings. Our work uses a novel technique for speech recognition using Melspectrogram and MFCC images. The results show that the ResNet18 classifier on speech-to-image converted data effectively identifies mispronunciations in Arabic speech with an accuracy of 99.015\% with Mel-Spectrogram images outperforming ResNet18 with MFCC images.
Sound,Artificial Intelligence
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the pronunciation barriers existing among Arabic - speaking children. Specifically, the author focuses on: 1. **The impact of pronunciation barriers**: Pronunciation barriers seriously affect children's communication abilities and academic progress, and have a significant impact on their social development and educational achievements. These problems may lead to a decline in self - esteem, dissatisfaction, a sense of shame, and even social withdrawal in severe cases. 2. **The particularity of Arabic**: Due to the complexity of Arabic phonetics and its differences from other languages, Arabic - speaking children have specific pronunciation barriers. These barriers may not disappear naturally as they grow older, so they need special help and support. 3. **The importance of early intervention**: Early intervention is crucial for the development of effective communication skills and language abilities. For this reason, the author proposes a new application named ArPA (Arabic Pronunciation Aid) aimed at helping Arabic - speaking children overcome pronunciation difficulties. ### Main contributions - **Dataset collection**: Collected an audio dataset containing correct and incorrect pronunciations of Arabic letters to ensure the reliability of model training. - **Research on traditional machine - learning algorithms**: Researched the performance of traditional machine - learning algorithms such as decision trees, KNN, and SVM in pronunciation classification and established benchmark performance. - **Application of deep - learning models**: Explored the use of image pre - trained deep - learning models (such as ResNet18) to process audio data and adopted transfer - learning techniques. - **Comprehensive evaluation**: Comprehensively evaluated the performance of all models through multiple metrics (such as accuracy, precision, recall, and F1 - score). - **Innovative application development**: Developed an application named ArPA, specifically targeting the pronunciation problems of Arabic - speaking children, including a diagnosis module and a treatment module. ### Conclusion By combining the diagnosis and treatment modules and using advanced deep - learning classifiers (such as ResNet18), ArPA can effectively identify pronunciation errors in Arabic - speaking children, demonstrating high accuracy, high recall, and high F1 - score. Future research will focus on expanding the dataset size, improving gamified treatment strategies, introducing more deep - learning architectures, and conducting long - term evaluations.