New Curriculum, New Chance -- Retrieval Augmented Generation for Lesson Planning in Ugandan Secondary Schools. Prototype Quality Evaluation

Simon Kloker,Herbertson Bukoli,Twaha Kateete

2024-08-14

Abstract:Introduction: Poor educational quality in Secondary Schools is still regarded as one of the major struggles in 21st century Uganda - especially in rural areas. Research identifies several problems, including low quality or absent teacher lesson planning. As the government pushes towards the implementation of a new curriculum, exiting lesson plans become obsolete and the problem is worsened. Using a Retrieval Augmented Generation approach, we developed a prototype that generates customized lesson plans based on the government-accredited textbooks. This helps teachers create lesson plans more efficiently and with better quality, ensuring they are fully aligned the new curriculum and the competence-based learning approach. Methods: The prototype was created using Cohere LLM and Sentence Embeddings, and LangChain Framework - and thereafter made available on a public website. Vector stores were trained for three new curriculum textbooks (ICT, Mathematics, History), all at Secondary 1 Level. Twenty-four lessons plans were generated following a pseudo-random generation protocol, based on the suggested periods in the textbooks. The lesson plans were analyzed regarding their technical quality by three independent raters following the Lesson Plan Analysis Protocol (LPAP) by Ndihokubwayo et al. (2022) that is specifically designed for East Africa and competence-based curriculums. Results: Evaluation of 24 lesson plans using the LPAP resulted in an average quality of between 75 and 80%, corresponding to "very good lesson plan". None of the lesson plans scored below 65%, although one lesson plan could be argued to have been missing the topic. In conclusion, the quality of the generated lesson plans is at least comparable, if not better, than those created by humans, as demonstrated in a study in Rwanda, whereby no lesson plan even reached the benchmark of 50%.

Computers and Society,Artificial Intelligence,Information Retrieval,Machine Learning

What problem does this paper attempt to address?

The paper attempts to address the issue of low-quality secondary education in Uganda, particularly in rural areas. Specifically, the study points out that the poor quality or absence of lesson plans by teachers is one of the main reasons for this problem. With the government implementing a new curriculum, existing lesson plans have become outdated, further exacerbating the issue. Therefore, the paper proposes a prototype tool based on the Retrieval-Augmented Generation (RAG) method, which can automatically generate customized lesson plans based on government-approved textbooks. This tool aims to help teachers create high-quality lesson plans more efficiently and ensure that these plans fully comply with the new curriculum requirements and competency-based teaching methods. Through this approach, the study aims to verify whether the lesson plans generated by RAG can meet the necessary standards required for implementation in Ugandan classrooms.