Abstract:Background: Clinical decisions are made based on Cochrane reviews, but the implementation of results of evidence syntheses such as Cochrane reviews is problematic if the evidence is not prepared consistently. All systematic reviews should assess the risk of bias (RoB) in included studies, and in Cochrane reviews, this is done by using Cochrane RoB tool. However, the tool is not necessarily applied according to the instructions. In this study, we aimed to determine the types of bias and their corresponding judgements noted in the 'other bias' domain of Cochrane RoB tool. Methods: We analyzed Cochrane reviews that included randomized controlled trials (RCTs) and extracted data regarding 'other bias' from the RoB table and accompanying support for the judgment. We categorized different types of other bias. Results: We analyzed 768 Cochrane reviews that included 11,369 RCTs. There were 602 (78%) Cochrane reviews that had 'other bias' domain in the RoB tool, and they included a total of 7811 RCTs. In the RoB table of 337 Cochrane reviews for at least one of the included trials it was indicated that no other bias was found and supporting explanations were inconsistently judged as low, unclear or high RoB. In the 524 Cochrane reviews that described various sources of other bias, there were 5762 individual types of explanations which we categorized into 31 groups. The judgments of the same supporting explanations were highly inconsistent. We found numerous other inconsistencies in reporting of sources of other bias in Cochrane reviews. Conclusion: Cochrane authors mention a wide range of sources of other bias in the RoB tool and they inconsistently judge the same supporting explanations. Inconsistency in appraising risk of other bias hinders reliability and comparability of Cochrane systematic reviews. Discrepant and erroneous judgments of bias in evidence synthesis may hinder implementation of evidence in routine clinical practice and reduce confidence in otherwise trustworthy sources of information. These results can help authors of Cochrane and non-Cochrane reviews to gain insight into various sources of other bias that can be found in trials, and also to help them avoid mistakes that were recognized in published Cochrane reviews.

Disagreements in risk of bias assessment for randomized controlled trials in hypertension-related Cochrane reviews

Reliability of the risk of bias assessment in randomized controlled trials for nursing: A cross‐sectional study

Agreement of Risk-of -Bias varied in systematic reviews on acupuncture and was associated with methodological quality

Towards the automatic risk of bias assessment on randomized controlled trials: A comparison of RobotReviewer and humans

Use and Reporting of Risk of Bias Tools in 825 Systematic Reviews of Acupuncture: a Cross-Sectional Study.

Risk of bias tool in systematic reviews/meta-analyses of acupuncture in Chinese journals

Association between risk-of-bias assessments and results of randomized trials in Cochrane reviews: the ROBES meta-epidemiologic study

Do Acupuncture Trials Have Lower Risk of Bias over the Last Five Decades? A Methodological Study of 4 715 Randomized Controlled Trials

Comparing machine and human reviewers to evaluate the risk of bias in randomized controlled trials

The judgement of biases included in the category "other bias" in Cochrane systematic reviews of interventions: a systematic survey

Risk of bias assessments and reporting quality of systematic reviews and randomized controlled trials examining acupuncture for depression: An overview and meta-epidemiology study

[Risk Related to Bias Assessment: (4) Revised Cochrane Risk of Bias Tool for Cluster-Randomized Control Trials (Rob2.0)].

Bias Risk Assessment: (3) Revised Cochrane Bias Risk Assessment Tool for Individual Randomized,cross-over Trials (Rob2.0)

[Risk on Bias Assessment: (2) Revised Cochrane Risk of Bias Tool for Individually Randomized, Parallel Group Trials (Rob2.0)].

Risk of Bias in Iranian Randomized Trials Included in Cochrane Reviews

RoB 2: a revised tool for assessing risk of bias in randomised trials

The ROBINS‐I and the NOS had similar reliability but differed in applicability: A random sampling observational studies of systematic reviews/meta‐analysis

Exploring the potential of Claude 2 for risk of bias assessment: Using a large language model to assess randomized controlled trials with RoB 2

Risk of Bias Assessment: (8) Risk of Bias in Systematic Review (ROBIS)

A case study of the informative value of risk of bias and reporting quality assessments for systematic reviews

Assessing risk of bias in a non‐randomized study