Abstract:This study examines whether there is any evidence of bias in two areas of common critique of open, non-anonymous peer review - and used in the post-publication, peer review system operated by the open-access scholarly publishing platform F1000Research. First, is there evidence of bias where a reviewer based in a specific country assesses the work of an author also based in the same country? Second, are reviewers influenced by being able to see the comments and know the origins of previous reviewer? Methods: Scrutinising the open peer review comments published on F1000Research, we assess the extent of two frequently cited potential influences on reviewers that may be the result of the transparency offered by a fully attributable, open peer review publishing model: the national affiliations of authors and reviewers, and the ability of reviewers to view previously-published reviewer reports before submitting their own. The effects of these potential influences were investigated for all first versions of articles published by 8 July 2019 to F1000Research. In 16 out of the 20 countries with the most articles, there was a tendency for reviewers based in the same country to give a more positive review. The difference was statistically significant in one. Only 3 countries had the reverse tendency. Second, there is no evidence of a conformity bias. When reviewers mentioned a previous review in their peer review report, they were not more likely to give the same overall judgement. Although reviewers who had longer to potentially read a previously published reviewer reports were slightly less likely to agree with previous reviewer judgements, this could be due to these articles being difficult to judge rather than deliberate non-conformity.

Does open peer review improve citation count? Evidence from a propensity score matching analysis of PeerJ

A Preliminary Investigation of the Effects of Open Review on Citation Impact

Does Opening Up Peer Review Benefit Science in Terms of Citations?

The Influence of Opening Up Peer Review on the Citations of Journal Articles

Does Open Identity of Peer Reviewers Positively Relate to Citations?

What Does Open Peer Review Bring to Scientific Articles? Evidence from PLoS Journals

Open Peer Review Correlates with Altmetrics but Not with Citations: Evidence from Nature Communications and PLoS One

Are Reviewer Scores Consistent with Citations?

Advantages and feasibility of double-blind peer-review for research articles

The Influence of Open Access on the Academic Purification Effect of Retracted Publications: A Causal Inference Analysis

Copublication improved the dissemination of Cochrane reviews and benefited copublishing journals: a retrospective cohort study

Does the use of open, non-anonymous peer review in scholarly publishing introduce bias? Evidence from the F1000 post-publication open peer review publishing model

Is Peer Review Effective?An Exploration Based on Efficient Market Hypothesis

An analysis of the effects of sharing research data, code, and preprints on citations

Do open access journal articles experience a citation advantage? Results and methodological reflections of an application of multiple measures to an analysis by WoS subject areas

Journal peer review: a bar or bridge? An analysis of a paper’s revision history and turnaround time, and the effect on citation

One-Bit In, Two-Bit Out: Network-Based Metrics of Papers Can Be Largely Improved by Including Only the External Citation Counts Without the Citation Relations

What Have We Learned from OpenReview?

A Bibliometic Study on Downloading and Citation of Open Access Papers

Binary black-hole evolutions of excision and puncture data

Propensity Score Analysis: Recent Debate and Discussion