Revised Cochrane risk-of-bias tool for randomized trials (RoB 2)

Julian PT Higgins, Jelena Savović, Matthew J Page, Jonathan AC Sterne

2019-08-22

Abstract:

What problem does this paper attempt to address?

Revised Cochrane risk of bias tool for randomized trials (RoB 2) Additional considerations for cluster-randomized trials (RoB 2 CRT); 2021

S Eldridge, MK Campbell, MJ Campbell, AK Drahota, B Giraudeau, BC Reeves, N Siegfried, JP Higgins

2021-01-01

Abstract:
RoB 2: a revised tool for assessing risk of bias in randomised trials

Jonathan A C Sterne,Jelena Savović,Matthew J Page,Roy G Elbers,Natalie S Blencowe,Isabelle Boutron,Christopher J Cates,Hung-Yuan Cheng,Mark S Corbett,Sandra M Eldridge,Jonathan R Emberson,Miguel A Hernán,Sally Hopewell,Asbjørn Hróbjartsson,Daniela R Junqueira,Peter Jüni,Jamie J Kirkham,Toby Lasserson,Tianjing Li,Alexandra McAleenan,Barnaby C Reeves,Sasha Shepperd,Ian Shrier,Lesley A Stewart,Kate Tilling,Ian R White,Penny F Whiting,Julian P T Higgins

DOI: https://doi.org/10.1136/bmj.l4898

2019-08-28

BMJ

Abstract:Assessment of risk of bias is regarded as an essential component of a systematic review on the effects of an intervention. The most commonly used tool for randomised trials is the Cochrane risk-of-bias tool. We updated the tool to respond to developments in understanding how bias arises in randomised trials, and to address user feedback on and limitations of the original tool.
RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ 366: l4898

JAC Sterne, J Savović, MJ Page, RG Elbers, NS Blencowe, I Boutron, CJ Cates, HY Cheng, MS Corbett, SM Eldridge, JR Emberson, MA Hernán, S Hopewell, A Hróbjartsson, DR Junqueira, P Jüni, JJ Kirkham, T Lasserson, T Li, A McAleenan, BC Reeves, S Shepperd, I Shrier, LA Stewart, K Tilling, IR White, PF Whiting, JPT Higgins

2019-01-01

Abstract:
Assessing risk of bias in a randomized trial

Julian PT Higgins, Jelena Savović, Matthew J Page, Roy G Elbers, Jonathan AC Sterne

2019-09-23

Abstract:This chapter details version 2 of the Cochrane risk‐of‐bias tool for randomized trials (RoB 2), the recommended tool for use in Cochrane Reviews. The RoB 2 tool is structured into domains through which bias might be introduced into the result. The chapter summarizes the main features of RoB 2 applied to individually randomized parallel‐group trials. It describes the process of undertaking an assessment using the RoB 2 tool, summarizes the important issues for each domain of bias, and deals with a list of the key differences between RoB 2 and the earlier version of the tool. The domains included in RoB 2 cover all types of bias that are currently understood to affect the results of randomized trials. These are bias arising from the randomization process; bias due to deviations from intended interventions; bias due to missing outcome data; bias in measurement of …
[Risk on Bias Assessment: (2) Revised Cochrane Risk of Bias Tool for Individually Randomized, Parallel Group Trials (Rob2.0)].

Z R Yang,F Sun,S Y Zhan

DOI: https://doi.org/10.3760/cma.j.issn.0254-6450.2017.09.028

2017-01-01

Abstract:This paper introduces the revised cochrane risk of bias tool for individually randomized, parallel group trials (RoB2.0), compates RoB2.0 and the previous version (RoB1.0). And illustrates the application of RoB2.0 for a published clinical trial. As a comprehensive tool, RoB2.0 provides more information on the risk of bias for evidence synthesis. RoB2.0 is still under development and it is suggested that the users should follow the updates of the developers in the future.
[Risk Related to Bias Assessment: (4) Revised Cochrane Risk of Bias Tool for Cluster-Randomized Control Trials (Rob2.0)].

X Z Lyu,F Sun,S Y Zhan

DOI: https://doi.org/10.3760/cma.j.issn.0254-6450.2018.02.020

2018-01-01

Abstract:This paper introduced the Revised Cochrane Risk of Bias Tool RoB2.0 for cluster-randomized control trials (CRCT) and compared RoB2.0 of CRCT with individually randomized, parallel group trials, and illustrated the application of RoB2.0 for CRCT in a published CRCT. Special signal questions were designed for CRCT according to its specialty that different from individually randomized, parallel group trials in RoB2.0 and also providing information on risk of bias about CRCT in systematic reviews for the synthesis of evidence.
Bias Risk Assessment: (3) Revised Cochrane Bias Risk Assessment Tool for Individual Randomized,cross-over Trials (Rob2.0)

S,F Sun,S Y Zhan

DOI: https://doi.org/10.3760/cma.j.issn.0254-6450.2017.10.028

2017-01-01

Abstract:This paper introduces the revised Cochrane bias risk assessment tool for individual randomized,cross-over trials (RoB2.0),comparing RoB2.0 of cross-over design with RoB2.0 of parallel design,and illustrating the application of RoB2.0 for cross-over trials in a published clinical trial.As a comprehensive tool specially designed for cross-over design,it provides more information on risk of bias in systematic reviews for the evidence synthesis.
A revised tool for assessing risk of bias in randomized trials

Julian PT Higgins, Jonathan AC Sterne, Jelena Savovic, Matthew J Page, Asbjørn Hróbjartsson, Isabelle Boutron, Barney Reeves, Sandra Eldridge

2016-01-01

Abstract:A revised tool for assessing risk of bias in randomized trials — Monash University Skip to main navigation Skip to search Skip to main content Monash University Home Monash University Logo Help & FAQ Home Profiles Research units Equipment Projects Outputs Prizes Activities Press/Media Search by expertise, name or affiliation A revised tool for assessing risk of bias in randomized trials Julian PT Higgins, Jonathan AC Sterne, Jelena Savovic, Matthew J Page, Asbjørn Hróbjartsson, Isabelle Boutron, Barney Reeves, Sandra Eldridge Research output: Contribution to journal › Comment / Debate › Other › peer-review Overview Original language English Pages (from-to) 29-31 Number of pages 3 Journal Cochrane Database of Systematic Reviews Volume 10 Issue number Suppl 1 Publication status Published - 2016 Externally published Yes Cite this APA Author BIBTEX Harvard Standard RIS Vancouver …
Evaluation of the Cochrane Collaboration’s tool for assessing the risk of bias in randomized trials: focus groups, online survey, proposed recommendations and their implementation

Jelena Savović, Laura Weeks, Jonathan AC Sterne, Lucy Turner, Douglas G Altman, David Moher, Julian PT Higgins

2014-12-01

Abstract:Background In 2008, the Cochrane Collaboration introduced a tool for assessing the risk of bias in clinical trials included in Cochrane reviews. The risk of bias (RoB) tool is based on narrative descriptions of evidence-based methodological features known to increase the risk of bias in trials. Methods To assess the usability of this tool, we conducted an evaluation by means of focus groups, online surveys and a face-to-face meeting. We obtained feedback from a range of stakeholders within The Cochrane Collaboration regarding their experiences with, and perceptions of, the RoB tool and associated guidance materials. We then assessed this feedback in a face-to-face meeting of experts and stakeholders and made recommendations for improvements and further developments of the RoB tool. Results The survey …
Revised Cochrane risk of bias tool for randomized trials (RoB 2.0): additional considerations for cluster-randomized trials

Sandra Eldridge, Marion Campbell, Michael Campbell, Amy Dahota, Bruno Giraudeau, Julian Higgins, Barney Reeves, Nandi Siegfried

2016-10-20

Abstract:In cluster-randomized trials, groups of individuals rather than individuals are randomized to different interventions. The groups may be, for example, schools, villages, medical practices or families. In some trials, individuals are allocated to interventions that are then applied to multiple parts of those individuals (for example, to both eyes or to several teeth), or repeated observations are made on a participant. If the analysis is by the individual units (for example, each tooth or each observation) then individual units are clustered within individuals in the same way that individuals are clustered within, for example, medical practices. However, the terminology used to describe the clusters (individuals) and cluster members (units) will often be different and trying to deal with both types of scenarios using one set of signalling questions is complex. In this tool we therefore focus on trials in which groups of individuals are randomized. This includes stepped-wedge trials in which randomization is by cluster, although there is a source of bias in these trials if they are analysed without adjustment for secular trends which is not directly covered in the tool (1).One of the main consequences of a cluster design is that participants within any one cluster often tend to respond in a similar manner, and thus their data can no longer be assumed to be independent of one another. It is important that the analysis of a cluster-randomized trial takes into account this issue. Unfortunately, many studies have in the past been incorrectly analysed as though the unit of allocation had been the individual participants. This is often referred to as a “unit-of-analysis error” because the unit of …
The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 343: d5928

JP Higgins, DG Altman, PC Gøtzsche, P Jüni, D Moher, AD Oxman, J Savovic, KF Schulz, L Weeks, JA Sterne

2011-01-01

Abstract:
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 355: i4919

Jonathan A Sterne, Miguel A Hernán, Barnaby C Reeves, Jelena Savović, Nancy D Berkman, Meera Viswanathan, D Henry, DG Altman, MT Ansari, I Boutron, JR Carpenter, AW Chan, R Churchill, JJ Deeks, A Hróbjartsson, J Kirkham, P Jüni, YK Loke, TD Pigott, CR Ramsay, D Regidor, HR Rothstein, L Sandhu, PL Santaguida, HJ Schünemann, B Shea, I Shrier, P Tugwell, L Turner, JC Valentine, H Waddington, E Waters, GA Wells, PF Whiting, JP Higgins

2016-01-01

Abstract:
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions

Jonathan AC Sterne,Miguel A Hernán,Barnaby C Reeves,Jelena Savović,Nancy D Berkman,Meera Viswanathan,David Henry,Douglas G Altman,Mohammed T Ansari,Isabelle Boutron,James R Carpenter,An-Wen Chan,Rachel Churchill,Jonathan J Deeks,Asbjørn Hróbjartsson,Jamie Kirkham,Peter Jüni,Yoon K Loke,Theresa D Pigott,Craig R Ramsay,Deborah Regidor,Hannah R Rothstein,Lakhbir Sandhu,Pasqualina L Santaguida,Holger J Schünemann,Beverly Shea,Ian Shrier,Peter Tugwell,Lucy Turner,Jeffrey C Valentine,Hugh Waddington,Elizabeth Waters,George A Wells,Penny F Whiting,Julian PT Higgins

DOI: https://doi.org/10.1136/bmj.i4919

2016-10-12

BMJ

Abstract:Non-randomised studies of the effects of interventions are critical to many areas of healthcare evaluation, but their results may be biased. It is therefore important to understand and appraise their strengths and weaknesses. We developed ROBINS-I (“Risk Of Bias In Non-randomised Studies - of Interventions”), a new tool for evaluating risk of bias in estimates of the comparative effectiveness (harm or benefit) of interventions from studies that did not use randomisation to allocate units (individuals or clusters of individuals) to comparison groups. The tool will be particularly useful to those undertaking systematic reviews that include non-randomised studies.
ROBIS: a new tool to assess risk of bias in systematic reviews was developed

Penny Whiting, Jelena Savović, Julian PT Higgins, Deborah M Caldwell, Barnaby C Reeves, Beverley Shea, Philippa Davies, Jos Kleijnen, Rachel Churchill

2016-01-01

Abstract:ObjectiveTo develop ROBIS, a new tool for assessing the risk of bias in systematic reviews (rather than in primary studies).Study Design and SettingWe used four-stage approach to develop ROBIS: define the scope, review the evidence base, hold a face-to-face meeting, and refine the tool through piloting.ResultsROBIS is currently aimed at four broad categories of reviews mainly within health care settings: interventions, diagnosis, prognosis, and etiology. The target audience of ROBIS is primarily guideline developers, authors of overviews of systematic reviews (“reviews of reviews”), and review authors who might want to assess or avoid risk of bias in their reviews. The tool is completed in three phases: (1) assess relevance (optional), (2) identify concerns with the review process, and (3) judge risk of bias. Phase 2 covers four domains through which bias may be introduced into a systematic review: study eligibility …
Association between risk-of-bias assessments and results of randomized trials in Cochrane reviews: the ROBES meta-epidemiologic study

Jelena Savović, Rebecca M Turner, David Mawdsley, Hayley E Jones, Rebecca Beynon, Julian PT Higgins, Jonathan AC Sterne

2018-05-01

Abstract:Flaws in the design of randomized trials may bias intervention effect estimates and increase between-trial heterogeneity. Empirical evidence suggests that these problems are greatest for subjectively assessed outcomes. For the Risk of Bias in Evidence Synthesis (ROBES) Study, we extracted risk-of-bias judgements (for sequence generation, allocation concealment, blinding, and incomplete data) from a large collection of meta-analyses published in the Cochrane Library (issue 4; April 2011). We categorized outcome measures as mortality, other objective outcome, or subjective outcome, and we estimated associations of bias judgements with intervention effect estimates using Bayesian hierarchical models. Among 2,443 randomized trials in 228 meta-analyses, intervention effect estimates were, on average, exaggerated in trials with high or unclear (versus low) risk-of-bias judgements for sequence …
The Cochrane Collaboration’s Tool for Assessing Risk of Bias in Randomised Trials

JP Higgins

2011-01-01

Abstract:Flaws in the design, conduct, analysis, and reporting of randomised trials can cause the effect of an intervention to be underestimated or overestimated. The Cochrane Collaboration's tool for assessing risk of bias aims to make the process clearer and more accurate
Frequency of use and adequacy of Cochrane risk of bias tool 2 in non‐Cochrane systematic reviews published in 2020: Meta‐research study

Andrija Babić,Ognjen Barcot,Tomislav Visković,Frano Šarić,Aleksandar Kirkovski,Ivana Barun,Zvonimir Križanac,Roshan Arjun Ananda,Yuli Viviana Fuentes Barreiro,Narges Malih,Daiana Anne‐Marie Dimcea,Josipa Ordulj,Ishanka Weerasekara,Matteo Spezia,Marija Franka Žuljević,Jelena Šuto,Luca Tancredi,Anđela Pijuk,Susanna Sammali,Veronica Iascone,Thilo von Groote,Tina Poklepović Peričić,Livia Puljak

DOI: https://doi.org/10.1002/jrsm.1695

2024-01-25

Research Synthesis Methods

Abstract:Risk of bias (RoB) assessment is essential to the systematic review methodology. The new version of the Cochrane RoB tool for randomized trials (RoB 2) was published in 2019 to address limitations identified since the first version of the tool was published in 2008 and to increase the reliability of assessments. This study analyzed the frequency of usage of the RoB 2 and the adequacy of reporting the RoB 2 assessments in non‐Cochrane reviews published in 2020. This meta‐research study included non‐Cochrane systematic reviews of interventions published in 2020. For the reviews that used the RoB 2 tool, we analyzed the reporting of the RoB 2 assessment. Among 3880 included reviews, the Cochrane RoB 1 tool was the most frequently used (N = 2228; 57.4%), followed by the Cochrane RoB 2 tool (N = 267; 6.9%). From 267 reviews that reported using the RoB 2 tool, 213 (79.8%) actually used it. In 26 (12.2%) reviews, erroneous statements were used to indicate the RoB 2 assessment. Only 20 (9.4%) reviews presented a complete RoB 2 assessment with a detailed table of answers to all signaling questions. The judgment of risk of bias by the RoB 2 tool was not justified by a comment in 158 (74.2%) reviews. Only in 33 (14.5%) of reviews the judgment in all domains was justified in the accompanying comment. In most reviews (81.7%), the RoB was inadequately assessed at the study level. In conclusion, the majority of non‐Cochrane reviews published in 2020 still used the Cochrane RoB 1 tool. Many reviews used the RoB 2 tool inadequately. Further studies about the uptake and the use of the RoB 2 tool are needed.

multidisciplinary sciences,mathematical & computational biology
Assessing risk of bias in included studies. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1. 0 (updated March 2011 …

JPT Higgins, DG Altman, JAC Sterne

2011-01-01

Abstract:
Assessing risk of bias in included studies. In: Higgins JPT, Churchill R, Chandler J, Cumpston MS (editors), Cochrane Handbook for Systematic Reviews of Interventions version 5 …

JP Higgins, DG Altman, JA Sterne

2017-01-01

Abstract:
Common challenges and suggestions for risk of bias tool development: a systematic review of methodological studies

Eve Tomlinson,Chris Cooper,Clare Davenport,Anne W.S. Rutjes,Mariska Leeflang,Sue Mallett,Penny Whiting

DOI: https://doi.org/10.1016/j.jclinepi.2024.111370

IF: 7.407

2024-04-26

Journal of Clinical Epidemiology

Abstract:Objective To review the findings of studies that have evaluated the design and/or usability of key risk of bias (RoB) tools for the assessment of RoB in primary studies, as categorised by the LATITUDES Network (a searchable library of RoB tools for evidence synthesis): PROBAST, RoB2, ROBINS-I, QUADAS-2, QUADAS-C, QUAPAS, ROBINS-E, and the COSMIN RoB checklist. Study design and setting Systematic review of methodological studies. We conducted a forward citation search from the primary report of each tool, to identify primary studies aiming to evaluate the design and/or usability of the tool. Two reviewers assessed studies for inclusion. We extracted tool features into Microsoft Word and used NVivo for document analysis, comprising a mix of deductive and inductive approaches. We summarised findings within each tool and explored common findings across tools. Results We identified 13 tool evaluations meeting our inclusion criteria: PROBAST (3); RoB2 (3); ROBINS-I (4); QUADAS-2 (3). We identified no evaluations for the other tools. Evaluations varied in clinical topic area, methodology, approach to bias assessment and tool user background. Some had limitations affecting generalisability. We identified common findings across tools for 6/14 themes: 1) challenging items (e.g. RoB2/ROBINS-I "deviations from intended interventions" domain), 2) overall RoB judgement (concerns with overall risk calculation in PROBAST/ROBINS-I), 3) tool usability (concerns about complexity), 4) time to complete tool (varying demands on time e.g. depending on number of outcomes assessed), 5) user agreement (varied across tools), and 6) recommendations for future use (e.g. piloting) and development (add intermediate domain answer to QUADAS-2/PROBAST; provide clearer guidance for all tools). Of the other eight themes, seven only had findings for the QUADAS-2 tool, limiting comparison across tools, and one ("re-organisation of questions") had no findings. Conclusion Evaluations of key RoB tools have posited common challenges and recommendations for tool use and development. These findings may be helpful to people using or developing RoB tools. Guidance is necessary to support the design and implementation of future RoB tool evaluations.

public, environmental & occupational health,health care sciences & services

Revised Cochrane risk-of-bias tool for randomized trials (RoB 2)

Revised Cochrane risk of bias tool for randomized trials (RoB 2) Additional considerations for cluster-randomized trials (RoB 2 CRT); 2021

RoB 2: a revised tool for assessing risk of bias in randomised trials

RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ 366: l4898

Assessing risk of bias in a randomized trial

[Risk on Bias Assessment: (2) Revised Cochrane Risk of Bias Tool for Individually Randomized, Parallel Group Trials (Rob2.0)].

[Risk Related to Bias Assessment: (4) Revised Cochrane Risk of Bias Tool for Cluster-Randomized Control Trials (Rob2.0)].

Bias Risk Assessment: (3) Revised Cochrane Bias Risk Assessment Tool for Individual Randomized,cross-over Trials (Rob2.0)

A revised tool for assessing risk of bias in randomized trials

Evaluation of the Cochrane Collaboration’s tool for assessing the risk of bias in randomized trials: focus groups, online survey, proposed recommendations and their implementation

Revised Cochrane risk of bias tool for randomized trials (RoB 2.0): additional considerations for cluster-randomized trials

The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 343: d5928

ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 355: i4919

ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions

ROBIS: a new tool to assess risk of bias in systematic reviews was developed

Association between risk-of-bias assessments and results of randomized trials in Cochrane reviews: the ROBES meta-epidemiologic study

The Cochrane Collaboration’s Tool for Assessing Risk of Bias in Randomised Trials

Frequency of use and adequacy of Cochrane risk of bias tool 2 in non‐Cochrane systematic reviews published in 2020: Meta‐research study

Assessing risk of bias in included studies. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1. 0 (updated March 2011 …

Assessing risk of bias in included studies. In: Higgins JPT, Churchill R, Chandler J, Cumpston MS (editors), Cochrane Handbook for Systematic Reviews of Interventions version 5 …

Common challenges and suggestions for risk of bias tool development: a systematic review of methodological studies