Textual analysis of teaching–learning evaluations in higher education: Deep learning and lexical investigation approaches
Henrique Faccin,Thiago Alexandro Nascimento de Andrade
DOI: https://doi.org/10.1016/j.eswa.2024.125982
IF: 8.5
2024-12-14
Expert Systems with Applications
Abstract:The evaluation of Brazilian universities, crucial for ensuring educational quality, is guided by the National System for Higher Education Evaluation (SINAES). In addition to meeting normative requirements, evaluations serve as tools for improvement, enabling strategic adjustments in teaching methods, faculty training, and institutional policies. This study addresses the complexity of the assessment, often based on open-ended questions that, although allowing detailed responses, pose a significant challenge in interpreting the texts of thousands of evaluative sentences. In this context, sentiment analysis emerges as a tool for extracting information from large datasets of texts. More than 23,000 sentences from the Evaluation of the Teaching–Learning Process at the Federal University of Santa Maria (UFSM) for the academic semesters of 2022 and 2023 were analyzed. Using deep learning techniques, such as the combination of recurrent neural networks with Long Short-Term Memory (LSTM), bidirectional LSTM and Gated Recurrent Unit (GRU) architectures, an accuracy of 84.1% was achieved in classifying evaluations such as praise , criticism , suggestion , and neutral . Furthermore, approximately 50% of the evaluations represented praise, and considering the lexical analysis, approximately 60% of the total evaluations were associated with positive sentiments. Among the identified emotions, the most frequent ones were also positive: trust (37%), joy (18%), and anticipation (10%). Finally, this study contributes to the understanding of educational evaluations. It highlights the importance of sentiment analysis as an auxiliary tool in effectively interpreting the responses, providing valuable insights for continuous improvement in higher education.
computer science, artificial intelligence,engineering, electrical & electronic,operations research & management science