How to measure research performance of single scientists? A proposal for an index based on scientific prizes: The Prize Winner Index (PWI)

Lutz Bornmann,Robin Haunschild
2023-03-31
Abstract:In this study, we propose a new index for measuring excellence in science which is based on collaborations (co-authorship distances) in science. The index is based on the Erdős number - a number that was introduced several years ago. We propose to focus with the new index on laureates of prestigious prizes in a certain field and to measure co-authorship distances between the laureates and other scientists. To exemplify and explain our proposal, we computed the proposed index in the field of quantitative science studies (PWIPM). The Derek de Solla Price Memorial Award (Price Medal, PM) is awarded to outstanding scientists in the field. We tested the convergent validity of the PWIPM. We were interested whether the indicator is related to an established bibliometric indicator: P(top 10%). The results show that the coefficients for the correlation between PWIPM and P(top 10%) are high (in cases when a sufficient number of papers have been considered for a reliable assessment of performance). Therefore, measured by an established indicator for research excellence, the new PWI indicator seems to be convergently valid and, therefore, might be a possible alternative for established (bibliometric) indicators - with a focus on prizes.
Digital Libraries,Physics and Society
What problem does this paper attempt to address?
The paper aims to address the issue of how to quantify an individual scientist's research performance. Existing evaluation methods primarily rely on citation metrics, but the authors propose a new index based on scientific awards: the Prize Winner Index (PWI), inspired by the mathematical community's Erdős number. PWI focuses on the collaboration relationships (co-author distance) between scientists who have received prestigious awards in a specific field and other scientists. By calculating the collaborative distance between awardees and their co-authors, the excellence of a scientist is measured. Taking the field of quantitative science studies as an example, the paper uses recipients of the Derek de Solla Price Memorial Award (Price Medal) to calculate the PWI PM. Through the analysis of different datasets, a high correlation between PWI PM and established citation metrics, such as the proportion of papers in the top 10% (P(top 10%)), was confirmed, indicating that PWI PM has convergent validity and could become an alternative to existing metrics. The study also discusses the advantages and potential issues of PWI PM compared to other citation-based metrics, such as the pressure on young scientists and the difficulty of choice due to an abundance of indicators. It points out that PWI PM emphasizes rewarding collaboration with elite scientists, suggesting that researchers who collaborate with elite scientists usually conduct high-quality research. In summary, the paper proposes a new method for evaluating scientists' performance, aiming to supplement or replace traditional citation metrics, providing a new perspective based on the collaboration network for scientific evaluation.