Abstract:Abstract Criteria are an essential component of any procedure for assessing merit. Yet, little is known about the criteria peers use to assess grant applications. In this systematic review we therefore identify and synthesize studies that examine grant peer review criteria in an empirical and inductive manner. To facilitate the synthesis, we introduce a framework that classifies what is generally referred to as ‘criterion’ into an evaluated entity (i.e., the object of evaluation) and an evaluation criterion (i.e., the dimension along which an entity is evaluated). In total, the synthesis includes 12 studies on grant peer review criteria. Two-thirds of these studies examine criteria in the medical and health sciences, while studies in other fields are scarce. Few studies compare criteria across different fields, and none focus on criteria for interdisciplinary research. We conducted a qualitative content analysis of the 12 studies and thereby identified 15 evaluation criteria and 30 evaluated entities, as well as the relations between them. Based on a network analysis, we determined the following main relations between the identified evaluation criteria and evaluated entities. The aims and outcomes of a proposed project are assessed in terms of the evaluation criteria originality , academic relevance , and extra-academic relevance . The proposed research process is evaluated both on the content level ( quality , appropriateness , rigor , coherence/justification ), as well as on the level of description ( clarity , completeness ). The resources needed to implement the research process are evaluated in terms of the evaluation criterion feasibility . Lastly, the person and personality of the applicant are assessed from a ‘psychological’ ( motivation , traits ) and a ‘sociological’ ( diversity ) perspective. Furthermore, we find that some of the criteria peers use to evaluate grant applications do not conform to the fairness doctrine and the ideal of impartiality. Grant peer review could therefore be considered unfair and biased. Our findings suggest that future studies on criteria in grant peer review should focus on the applicant, include data from non-Western countries, and examine fields other than the medical and health sciences.

Past Performance as Predictor of Successful Grant Applications: A Case Study

Past performance, peer review, and project selection: A case study in the social and behavioral sciences

Investigating Academic Graph‐Based Factors Behind Funding Success in National Institutes of Health

A meta-evaluation of scientific research proposals: Different ways of comparing rejected to awarded applications

Predictors of applying for and winning an ERC Proof-of-Concept grant: An automated machine learning model

Correction: Past Performance, Peer Review, and Project Selection. Research Evaluation 18 (2009) Peter Van Den Besselaar, Loet Leydesdorff

Assessment of potential bias in research grant peer review in Canada

Criteria for assessing grant applications: a systematic review

Gender differences in research grant allocation -- a mixed picture

Reviewers' ratings and bibliometric indicators: hand in hand when assessing over research proposals?

An experimental study of simulated grant peer review: Gender differences and psychometric characteristics of proposal scores

What Factors of the Applicants Are Influencing the Performance of Research Science Education? A Back-chaining Evaluation Based on the Data of China National Science Foundation

Latent Markov modelling applied to grant peer review

Rethinking the Funding Line at the Swiss National Science Foundation: Bayesian Ranking and Lottery

Toward predicting research proposal success

Lotteries make science fairer

Designing Scientific Grants

Competition for funding as an indicator of research competitiveness

Contest models highlight inherent inefficiencies of scientific funding competitions

Gender differences in Dutch research funding over time: A statistical investigation of the innovation scheme 2012–2021

Rethinking Resource Allocation in Science