Abstract:The peer review system has been traditionally challenged due to its many limitations especially for allocating funding. Bibliometric indicators may well present themselves as a complement. Objective: We analyze the relationship between peers' ratings and bibliometric indicators for Spanish researchers in the 2007 National R&D Plan for 23 research fields. We analyze peers' ratings for 2333 applications. We also gathered principal investigators' research output and impact and studied the differences between accepted and rejected applications. We used the Web of Science database and focused on the 2002-2006 period. First, we analyzed the distribution of granted and rejected proposals considering a given set of bibliometric indicators to test if there are significant differences. Then, we applied a multiple logistic regression analysis to determine if bibliometric indicators can explain by themselves the concession of grant proposals. 63.4% of the applications were funded. Bibliometric indicators for accepted proposals showed a better previous performance than for those rejected; however the correlation between peer review and bibliometric indicators is very heterogeneous among most areas. The logistic regression analysis showed that the main bibliometric indicators that explain the granting of research proposals in most cases are the output (number of published articles) and the number of papers published in journals that belong to the first quartile ranking of the Journal Citations Report. Bibliometric indicators predict the concession of grant proposals at least as well as peer ratings. Social Sciences and Education are the only areas where no relation was found, although this may be due to the limitations of the Web of Science's coverage. These findings encourage the use of bibliometric indicators as a complement to peer review in most of the analyzed areas.

The choice: evaluating and selecting scientific proposals

On choosing meaningful research projects in the natural sciences

Design and validation of a decision support checklist for efficient resource allocation in research projects during proposal preparation

Inefficiency in selecting products for submission to national research assessment exercises

A meta-evaluation of scientific research proposals: Different ways of comparing rejected to awarded applications

Contest models highlight inherent inefficiencies of scientific funding competitions

Designing Scientific Grants

Organizational Selection of Innovation

Past performance, peer review, and project selection: A case study in the social and behavioral sciences

Rethinking Resource Allocation in Science

Assessing the Effectiveness of Quantitative and Qualitative Methods for R&D Project Proposal Evaluations

Past Performance as Predictor of Successful Grant Applications: A Case Study

Flipping the grant application review process

Reviewers' ratings and bibliometric indicators: hand in hand when assessing over research proposals?

A dispersion-driven method for grant and proposal allocation

Evaluating innovation. Part 1: The concept of progressive scholarly acceptance

A Decision Support System for Public Research Organizations Participating in National Research Assessment Exercises

Competition for funding as an indicator of research competitiveness

A systematic review of creativity evaluation and creativity selection measurement tasks.

Theoretical research without projects

The Importance of Separating Appropriateness into Impact and Feasibility for the Psychology of Creativity