Theory-based evaluation of the impact of Master's programmes in development studies: Insights from a mixed-methods and multicultural alumni action research project
Nathalie Holvoet,Wanda Casten,Eshetu Woldeyohannes Demissie,Sara Dewachter,Marian Kaye C Gamboa,Tewelde Gebremariam Adhanom,Abdurahman Hamza Ibrahim,Hezron Makundi,Grachel Manguni,Georgina Angela Manyuru,Nicholas Mugabi,Yahya E Nawanda,Chau Hoai Nguyen,William Amos Pallangyo,Rose Pascual-Villar,Luz Prado Saavedra,Mariluz Torres Salgado,Nash Tysmans,Anh Ngoc Vu,Eva Wuyts,Zerihun Berhane Weldegebriel
DOI: https://doi.org/10.1016/j.evalprogplan.2023.102228
Abstract:To understand the impact of the international Master's programmes offered at the Institute of Development Policy (University of Antwerp), a theory-based evaluation was undertaken. In the first phase, a Theory of Change (ToC) was elaborated, distinguishing between three levels of impact (individual, organisational and societal), four learning dimensions (knowledge, skills, attitudes and networks) and five implicit pathways (change agent, social network, widening access, academic diversity, international understanding). Given the multifaceted and vague nature of the 'impact' concept under study, we selected an international, gender-balanced, multi-sectoral team of alumni researchers who fostered inclusiveness of different perspectives, at the same time capitalising on their depth of understanding, having gone through the study experience themselves. A mixed-methods approach was adopted to validate the ToC, combining a Most Significant Change approach, categorisation and text analysis of 101 alumni impact stories. Our findings demonstrate the importance of the three levels of impact and four learning dimensions in capturing graduate impact. While the impact stories confirmed the dominant 'change agent' pathway, they also hinted at the importance of hybrid complementary configurations of pathways to fully grasp how impact materialises.