The quality of COVID-19 systematic reviews during the coronavirus 2019 pandemic: an exploratory comparison

Kevin T. McDermott,Mark Perry,Willemijn Linden,Rachel Croft,Robert Wolff,Jos Kleijnen
DOI: https://doi.org/10.1186/s13643-024-02552-x
2024-05-09
Systematic Reviews
Abstract:The unprecedented volume and speed at which COVID-19-related systematic reviews (SRs) may have been produced has raised questions regarding the quality of this evidence. It is feasible that pandemic-related factors may have led to an impairment in quality (reduced internal validity, increased risk of bias [RoB]). This may have serious implications for decision-making related to public health and individual healthcare.
medicine, general & internal
What problem does this paper attempt to address?
This paper focuses on the quality issues of systematic reviews (SRs) published during the COVID-19 pandemic. Researchers compared the quality differences between COVID-19-related SRs and non-COVID-19-related SRs, as well as between SRs published during the pandemic and those published before. Background: The rapid spread of COVID-19 and the emergence of a large volume of related research have raised concerns about the quality of this evidence, which may affect public health decision-making and individual medical care. Objectives: 1. Compare the quality of COVID-19-related SRs and non-COVID-19-related SRs during the pandemic. 2. Compare the quality of all SRs (regardless of topic) published during the pandemic with those published before. Methods: Researchers used the ROBIS tool to assess the quality of all COVID-19-related and non-COVID-19-related SRs in the KSR Evidence database and compared the SRs published during the pandemic with those published before. Results: 1. There were no significant differences in quality between COVID-19-related SRs and non-COVID-19-related SRs published during the pandemic. Both showed low quality, with approximately 10%-11% rated as low risk of bias. 2. The overall quality of SRs published during the pandemic was lower than that of those published before, with the proportion of low risk of bias decreasing from 36% before the pandemic to 11%. Conclusion: The quality of both COVID-19-related and non-COVID-19-related SRs during the pandemic was low, and the overall quality of SRs published during the pandemic was generally lower than before. This suggests the need to make efforts to improve the quality and rigor of SRs to ensure their reliability as a basis for decision-making.