Abstract:Peer review is the "gold standard" for evaluating journal and conference papers, research proposals, on-going projects and university departments. However, it is widely believed that current systems are expensive, conservative and prone to various forms of bias. One form of bias identified in the literature is "social bias" linked to the personal attributes of authors and reviewers. To quantify the importance of this form of bias in modern peer review, we analyze three datasets providing information on the attributes of authors and reviewers and review outcomes: one from Frontiers - an open access publishing house with a novel interactive review process, and two from Spanish and international computer science conferences, which use traditional peer review. We use a random intercept model in which review outcome is the dependent variable, author and reviewer attributes are the independent variables and bias is defined by the interaction between author and reviewer attributes. We find no evidence of bias in terms of gender, or the language or prestige of author and reviewer institutions in any of the three datasets, but some weak evidence of regional bias in all three. Reviewer gender and the language and prestige of reviewer institutions appear to have little effect on review outcomes, but author gender, and the characteristics of author institutions have moderate to large effects. The methodology used cannot determine whether these are due to objective differences in scientific merit or entrenched biases shared by all reviewers.

Editor bias and diversifying psychology: Commentary on Sharpe (2024).

Editor bias and transparency in psychology's open science era.

To reduce editor bias and increase diversity and transparency, editors must be motivated: Commentary on Sharpe (2024).

A decolonized science requires bigger, bolder, and less incremental change: Commentary on Sharpe (2024).

On editor bias and transparency: Reply to Cowan (2024), King (2024), and Thurston and Noor (2024).

The Effects of Editorial-Board Diversity on Race Scholars and Their Scholarship: A Field Experiment

Psychology needs more diversity at the level of Editor-in-Chief

Dealing With Diversity in Psychology: Science and Ideology

Diversity Is Diverse: Social Justice Reparations and Science

Toward the advancement of equity in scientific publishing.

Citation bias, diversity, and ethics

Dealing With Diversity in Psychology: Science or Ideology?

Reducing racial bias in scientific communication: Journal policies and their influence on reporting racial demographics.

Liberal Bias on Psychology: A Comment on Bartels (2023)

Peer review perpetuates barriers for historically excluded groups

Racial Inequality in Psychological Research: Trends of the Past and Recommendations for the Future

Understanding bias through diverse lenses

Social psychological research on gender, sexuality, and relationships: reflections on contemporary scientific and cultural challenges

Pushing Back Against the Microaggression Pushback in Academic Psychology: Reflections on a Concept-Creep Paradox

Pride-and-Prejudice Perspectives of Marginalization Can Advance Science and Society

Personal attributes of authors and reviewers, social bias and the outcomes of peer review: a case study