A Reviewer-Reputation Ranking Algorithm to Identify High-Quality Papers During the Review Process

Fujuan Gao,Enrico Maria Fenoaltea,Pan Zhang,An Zeng
DOI: https://doi.org/10.1016/j.eswa.2024.123551
IF: 8.5
2024-01-01
Expert Systems with Applications
Abstract:With the exponential growth in the number of academic researchers, it is crucial for editors of scientific journals to identify the highest-quality papers. While several measures exist to evaluate a paper’s impact post-publication, the challenge of determining the potential impact of a manuscript during the review process remains an understudied issue. In this paper, we propose a reviewer-reputation ranking algorithm to identify high-quality papers based on paper citations, where a reviewer’s reputation is computed from the correlation between their past ratings and the current number of citations received by the papers they have evaluated. During the review process, reviewers with high reputation scores are given more weight to determine the quality of papers. We test the algorithm on an artificial network with 200 reviewers and 600 papers, as well as on the American Physical Society (APS) data set, including in the analysis 308,243 papers and 274,154 mutual citations. We compare our approach with two existing methods, demonstrating that our algorithm significantly outperforms the others in identifying manuscripts with the highest quality. Our findings can help improve the impact of scientific journals, thereby contributing to academic and scientific progress.
What problem does this paper attempt to address?