Abstract:On the behest of the Office of Science and Technology Policy in the White House, six institutions, including ours, have created an open research dataset called COVID-19 Research Dataset (CORD-19) to facilitate the development of question-answering systems that can assist researchers in finding relevant research on COVID-19. As of May 27, 2020, CORD-19 includes more than 100,000 open access publications from major publishers and PubMed as well as preprint articles deposited into medRxiv, bioRxiv, and arXiv. Recent years, however, have also seen question-answering and other machine learning systems exhibit harmful behaviors to humans due to biases in the training data. It is imperative and only ethical for modern scientists to be vigilant in inspecting and be prepared to mitigate the potential biases when working with any datasets. This article describes a framework to examine biases in scientific document collections like CORD-19 by comparing their properties with those derived from the citation behaviors of the entire scientific community. In total, three expanded sets are created for the analyses: 1) the enclosure set CORD-19E composed of CORD-19 articles and their references and citations, mirroring the methodology used in the renowned “A Century of Physics” analysis; 2) the full closure graph CORD-19C that recursively includes references starting with CORD-19; and 3) the inflection closure CORD-19I, that is, a much smaller subset of CORD-19C but already appropriate for statistical analysis based on the theory of the scale-free nature of the citation network. Taken together, all these expanded datasets show much smoother trends when used to analyze global COVID-19 research. The results suggest that while CORD-19 exhibits a strong tilt toward recent and topically focused articles, the knowledge being explored to attack the pandemic encompasses a much longer time span and is very interdisciplinary. A question-answering system with such expanded scope of knowledge may perform better in understanding the literature and answering related questions. However, while CORD-19 appears to have topical coverage biases compared to the expanded sets, the collaboration patterns, especially in terms of team sizes and geographical distributions, are captured very well already in CORD-19 as the raw statistics and trends agree with those from larger datasets.

COVID-19 Multidimensional Kaggle Literature Organization

COVID-19 Kaggle Literature Organization

Mapping the vast landscape of multisystem complications of COVID-19: Bibliometric analysis

COVID-19 Knowledge Graph: Accelerating Information Retrieval and Discovery for Scientific Literature

Visualising COVID-19 Research

COVID-19 Literature Topic-Based Search via Hierarchical NMF

Coronavirus Knowledge Graph: A Case Study

COVID-19 and Big Data: Multi-faceted Analysis for Spatio-temporal Understanding of the Pandemic with Social Media Conversations

Exploring the Effect of Social Media and Spatial Characteristics During the COVID-19 Pandemic in China

Mitigating Biases in CORD-19 for Analyzing COVID-19 Literature

Enhanced Sentiment Analysis and Topic Modeling During the Pandemic Using Automated Latent Dirichlet Allocation

Collaboration in the Time of COVID: A Scientometric Analysis of Multidisciplinary SARS-CoV-2 Research

A Glimpse of the First Eight Months of the COVID-19 Literature on Microsoft Academic Graph: Themes, Citation Contexts, and Uncertainties

COVID-19 research progress: Bibliometrics and visualization analysis

Impact of COVID-19 research: a study on predicting influential scholarly documents using machine learning and a domain-independent knowledge graph

Target specific mining of COVID-19 scholarly articles using one-class approach

Understanding the temporal evolution of COVID-19 research through machine learning and natural language processing

Exploring the evolution of research topics during the COVID-19 pandemic

On the Application of Advanced Machine Learning Methods to Analyze Enhanced, Multimodal Data from Persons Infected with COVID-19

A curated collection of COVID-19 online datasets

CORACLE (COVID-19 liteRAture CompiLEr): A platform for efficient tracking and extraction of SARS-CoV-2 and COVID-19 literature, with examples from post-COVID with respiratory involvement