Abstract:Purpose: The growing public interest in vocational education and training (VET), most recently since the economic crisis of 2007/2008, has led to an exponential increase in articles with a vocational focus, underscoring the need for review studies for the purposes of systematic knowledge aggregation, clarification and interpretation. We assume that review studies follow the same minimum standards as other research methods: the review must be at leastreproducible and thus the results verifiable or falsifiable. So far, however, the review methods used in VET research have not been investigated. Our purpose is to review the review procedures and methods used in published reviews of VET research to identify their current methodological quality. Approach: To classify the review studies, we initially developed a conceptual framework to distinguish different types of reviews. We then developed a methodological framework to assess the review methods used. Overall, to accelerate the review process, our review of reviews (or umbrella review) followed the rapid review approach: we limited our search to reviews in English published between 2014 and 2019 in peer-reviewed journals with a substantial VET focus and indexed in Scopus and/or Web of Science. Therefore, we did not examine all existing reviews in the field of VET research. Rather, our specific focus was on a core sector of scientific research: peer-reviewed articles in curated databases. Furthermore, we concentrated on the review procedures and methods used, not on the content of the reviews.Findings: We identified nine journals with a substantial VET focus, yielding a total of 1,283 published articles between 2014 and 2019, of which only 19 articles (1.48%) were literature reviews. Of these 19 reviews, six were excluded from our umbrella review because of unclear methodological procedures. Based on the review typology we developed, five of the remaining 13 reviews were conceptual in nature, four were scoping reviews, three were evidence-oriented, and one was critical in nature. None of the reviews examined focused on meta-syntheses, research methods or meta-analyses. In total, this resulted in current review gaps with respect to theory generation (meta-synthesis), practice of theory elaboration and testing (methodological review) and the determination of overall effects across single studies (meta-analysis). Finally, our examination of the reviews showed that their scope was mostly clearly presented. However, with regard to the process steps 'data selection' and 'data processing', only a few reviews fully met the requirements of the methodological framework. C onclusion: Our review leads to four conclusions. 1) More systematic syntheses are needed because there is a substantial quantitative gap in review research. 2) In particular, there is a need for review studies with a focus on meta-synthesis, research methods and meta-analysis, as there is a current gap in these areas. 3) Reviews should be based on a review methodology with transparent and reproducible methods and verifiable or falsifiable results. The high number of subjective syntheses with unclear review procedures indicates that this mindset is not yet fully established in the field of VET research. 4) In the studies examined, there is a high degree of heterogeneity regarding to the accuracy and completeness of the methodological steps and data. The conceptual and methodological frameworks developed for the analysis can serve as guidelines for the conduct of reviews, and thus, the frameworks could contribute to the further development of the methodological basis of reviews.

Umbrella Review as an Emerging Approach of Evidence Synthesis in Health Sciences: A Bibliometric Analysis

Ten simple rules for conducting umbrella reviews

Bibliometric study of ‘overviews of systematic reviews’ of health interventions: evaluation of prevalence, citation and impact factor

Bibliometric study of ‘overviews of systematic reviews’ of health interventions: Evaluation of prevalence, citation and journal impact factor

Characterising umbrella reviews in paediatrics: A systematic analysis

[Evidence syntheses in public health: An overview]

Need for evidence synthesis for quality control of healthcare decision-making

metaumbrella: the first comprehensive suite to perform data analysis in umbrella reviews with stratification of the evidence

An Umbrella Review of Systematic Reviews on Food Choice and Nutrition Published between 2017 and-2019

Social Networks as a Tool for Evidence-Based Health Education: Umbrella Review

Epidemiology and reporting characteristics of overviews of reviews of healthcare interventions published 2012-2016: protocol for a systematic review

Umbrella review: Impact of registered nurses' continuing professional education informed by contemporary reviews of literature

Evidence for curricular and instructional design approaches in undergraduate medical education: An umbrella review

Use and impact of clinical pathways across various healthcare settings: A protocol for an umbrella review of global evidence

A review of the quantitative effectiveness evidence synthesis methods used in public health intervention guidelines

Which public health interventions are effective in reducing morbidity, mortality and health inequalities from infectious diseases amongst children in low- and middle-income countries (LMICs): An umbrella review

Climate change, biodiversity loss, and Indigenous Peoples’ health and wellbeing: a systematic umbrella review protocol

Design and analysis of umbrella trials: Where do we stand?

Universal Health Coverage – Exploring the What, How, and Why Using Realistic Review

Umbrella review: Methodological review of reviews published in peer-reviewed journals with a substantial focus on vocational education and training research