Abstract:Background: Journal impact factor (IF) is the leading method of scholarly assessment in today's research world, influencing where scholars submit their research and where funders distribute their resources. COVID-19, one of the most serious health crises, resulted in an unprecedented surge of publications across all areas of knowledge. An important question is whether COVID-19 affected the gold standard of scholarly assessment . Objective: In this paper, we aimed to comprehensively compare the productivity trends of COVID-19 and non–COVID-19 literature as well as track their evolution and scholarly impact across 3 consecutive calendar years. Methods: We took as an example 6 high-impact medical journals (Annals of Internal Medicine [Annals], The British Medical Journal [The BMJ], Journal of the American Medical Association [JAMA], The Lancet, Nature Medicine [NatMed], and The New England Journal of Medicine [NEJM]) and searched the literature using the Web of Science database for manuscripts published between January 1, 2019, and December 31, 2021. To assess the effect of COVID-19 and non–COVID-19 literature in their scholarly impact, we calculated their annual IFs and percentage changes. Thereafter, we estimated the citation probability of COVID-19 and non–COVID-19 publications along with their rates of publication and citation by journal. Results: A significant increase in IF change for manuscripts including COVID-19 published from 2019 to 2020 ( P =.002; Annals: 283%; The BMJ: 199%; JAMA: 208%; The Lancet: 392%; NatMed: 111%; and NEJM: 196%) and to 2021 ( P =.007; Annals: 41%; The BMJ: 90%; JAMA: 6%; The Lancet: 22%; NatMed: 53%; and NEJM: 72%) was seen, against non–COVID-19 ones. The likelihood of highly cited publications was significantly increased in COVID-19 manuscripts between 2019 and 2021 (Annals: z =3.4, P <.001; The BMJ: z =4.0, P <.001; JAMA: z =3.8, P <.001; The Lancet: z =3.5, P <.001; NatMed: z =5.2, P <.001; and NEJM: z =4.7, P <.001). The publication and citation rates of COVID-19 publications followed a positive trajectory, as opposed to non–COVID-19. The citation rate for COVID-19 publications peaked by the second quarter of 2020 while that of the publication rate approximately a year later. Conclusions: The rapid surge of COVID-19 publications emphasized the capacity of scientific communities to respond against a global health emergency, yet inflated IFs create ambiguity as benchmark tools for assessing scholarly impact. The immediate implication is a loss in value of and trust in journal IFs as metrics of research and scientific rigor perceived by academia and society. Loss of confidence toward procedures employed by highly reputable publishers may incentivize authors to exploit the publication process by monopolizing their research on COVID-19 and encourage them to publish in journals of predatory behavior.

The significant yet short-term influence of research covidization on journal citation metrics

Citation advantage of COVID-19 related publications

The impact of COVID-19 on global health journals: an analysis of impact factor and publication trends

How COVID-19 Affected the Journal Impact Factor of High Impact Medical Journals: Bibliometric Analysis

COVID-19: a Disruptive Impact on the Knowledge Support of References

Preprints vs. Journal Articles: Citation Impact in COVID-19 Research

Auditing citation polarization during the early COVID-19 pandemic

Explosive increase and decrease in articles, citations, impact factor, and immediacy index during the COVID-19 pandemic: a bibliometric study

Impact of COVID-19 Pandemic on Biomedical Publications and Their Citation Frequency

The impact of the COVID-19 pandemic on scientific research in the life sciences

The effect of the rapid growth of covid-19 publications on citation indicators

COVID-19 Pandemic and Impact on Research Publications in Critical Care

Publishing trends of journals and articles in Journal Citation Reports during the COVID-19 pandemic: a descriptive study

A scientometric analysis of the effect of COVID-19 on the spread of research outputs

The impact of the COVID pandemic on publication disciplines among major general medical journals: A bibliometric analysis using natural language processing and traditional statistics

Public Health Crisis Management Caused by COVID‐19: A Scientometrics Review

The Impact of Preprints on the Citations of Journal Articles Related to COVID-19

Evaluating the dependability of reference-driven citation forecasts amid the COVID-19 pandemic: A bibliometric analysis across diverse journals

Global analysis of the COVID-19 research landscape and scientific impact

Mapping the vast landscape of multisystem complications of COVID-19: Bibliometric analysis

The impact of COVID-19-related chronic disease is gradually emerging: discovery and trends from a bibliometric analysis