Abstract:Peer review (e.g., grading assignments in Massive Open Online Courses (MOOCs), academic paper review) is an effective and scalable method to evaluate the products (e.g., assignments, papers) of a large number of agents when the number of dedicated reviewing experts (e.g., teaching assistants, editors) is limited. Peer review poses two key challenges: 1) identifying the reviewers' intrinsic capabilities (i.e., adverse selection) and 2) incentivizing the reviewers to exert high effort (i.e., moral hazard). Some works in mechanism design address pure adverse selection using one-shot matching rules, and pure moral hazard was addressed in repeated games with exogenously given and fixed matching rules. However, in peer review systems exhibiting both adverse selection and moral hazard, one-shot or exogenous matching rules do not link agents' current behavior with future matches and future payoffs, and as we prove, will induce myopic behavior (i.e., exerting the lowest effort) resulting in the lowest review quality. In this paper, we propose for the first time a solution that simultaneously solves adverse selection and moral hazard. Our solution exploits the repeated interactions of agents, utilizes ratings to summarize agents' past review quality, and designs matching rules that endogenously depend on agents' ratings. Our proposed matching rules are easy to implement and require no knowledge about agents' private information (e.g., their benefit and cost functions). Yet, they are effective in guiding the system to an equilibrium where the agents are incentivized to exert high effort and receive ratings that precisely reflect their review quality. Using several illustrative examples, we quantify the significant performance gains obtained by our proposed mechanism as compared to existing one-shot or exogenous matching rules.

Another Look at Job Design with Conflicting Tasks

Job Design with Sequential Tasks and Outcome Externalities Revisited

Scheduling Games with Potential Penalties on the Move of Jobs

Dynamic Contract Design in the Presence Of Double Moral Hazard

The Cooperative Consequences of Contests

Payoff Mechanism Design for Coordination in Multi-Agent Task Allocation Games

Contracting with Cost Synergies: Continuous-Time Double-Sided Moral Hazard

Designing organizations for bottom-up task allocation: The role of incentives

Optimal contracts under competition when uncertainty from adverse selection and moral hazard are present

To Leave or to Stay? Research on the Double-Edged Sword Effect of Illegitimate Tasks on Employee Turnover Intention

RISK-SHARING MATCHING AND MORAL HAZARD

Contracting in a Continuous-Time Model with Three-Sided Moral Hazard and Cost Synergies

Promotion and demotion contests

Competition Alleviates Present Bias in Task Completion

Returning a Treasure: Revisiting the Asymmetric Matching Pennies Contradiction

Incentive Design in Peer Review: Rating and Repeated Endogenous Matching

Challenge or Threat? Examining When and How Employees React Positively and Negatively to Coworkers' Job Crafting

Beyond top‐down and bottom‐up work redesign: Customizing job content through idiosyncratic deals

The Ambivalence of Job Autonomy and the Role of Job Design Demands

Employee Well‐being Attribution and Job Change Intentions: the Moderating Effect of Task Idiosyncratic Deals

Stackelberg risk preference design