Abstract:Reviewer assignment is critical to peer review systems, such as peer-reviewed research conferences or peer-reviewed funding applications, and its effectiveness is a deep concern of all academics. However, there are some problems in existing peer review systems during reviewer assignment. For example, some of the reviewers are much more stringent than others, leading to an unfair final decision, i.e., some submissions (i.e., papers or applications) with better quality are rejected. In this paper, we propose a context-aware reviewer assignment for trust enhanced peer review. More specifically, in our approach, we first consider the research area specific expertise of reviewers, and the institution relevance and co-authorship between reviewers and authors, so that reviewers with the right expertise are assigned to the corresponding submissions without potential conflict of interest. In addition, we propose a novel cross-assignment paradigm, and reviewers are cross-assigned in order to avoid assigning a group of stringent reviewers or a group of lenient reviewers to the same submission. More importantly, on top of them, we propose an academic CONtext-aware expertise relevanCe oriEnted Reviewer cross-assignmenT approach (CONCERT), which aims to effectively estimate the "true" ratings of submissions based on the ratings from all reviewers, even though no prior knowledge exists about the distribution of stringent reviewers and lenient reviewers. The experiments illustrate that compared with existing approaches, our proposed CONCERT approach can less likely assign more than one stringent reviewers or lenient reviewers to a submission simultaneously and significantly reduce the influence of ratings from stringent reviewers and lenient reviewers, leading to trust enhanced peer review and selection, no matter what kind of distributions of stringent reviewers and lenient reviewers are.

Time-Aware And Topic-Based Reviewer Assignment

PaRe: A Paper-Reviewer Matching Approach Using a Common Topic Space

Submission-Aware Reviewer Profiling for Reviewer Recommender System

Who Should Review This Change?: Putting Text and File Location Analyses Together for More Accurate Recommendations

Expert Team Finding for Review Assignment

Code Reviewer Recommendation in Tencent

Recommending Code Reviewers for Proprietary Software Projects: A Large Scale Study

Context-Aware Reviewer Assignment For Trust Enhanced Peer Review

Code Reviewer Recommendation in Tencent: Practice, Challenge, and Direction*

A decision support approach for assigning reviewers to proposals

Solving Reviewer Assignment Problem in Software Peer Review: an Approach Based on Preference Matrix and Asymmetric TSP Model

Chain-of-Factors Paper-Reviewer Matching

Reviewer assignment problem: A scoping review

Research Analytics for Reviewer Recommendation

A Novel Classification Method for Paper-Reviewer Recommendation

Into the Unknown: Assigning Reviewers to Papers with Uncertain Affinities

Decision Support System for Assignment of Conference Papers to Reviewers

Choosing a Better Moment to Assign Reviewers in Peer Assessment: the Earlier the Better, or the Later the Better?

An efficient ontology-based topic-specific article recommendation model for best-fit reviewers

An Automatic Paper-Reviewer Recommendation Algorithm Based on Depth and Breadth

A Code Reviewer Assignment Model Incorporating the Competence Differences and Participant Preferences