Abstract:This study explores the extent to which bibliometric indicators based on counts of highly-cited documents could be affected by the choice of data source. The initial hypothesis is that databases that rely on journal selection criteria for their document coverage may not necessarily provide an accurate representation of highly-cited documents across all subject areas, while inclusive databases, which give each document the chance to stand on its own merits, might be better suited to identify highly-cited documents. To test this hypothesis, an analysis of 2515 highly-cited documents published in 2006 that Google Scholar displays in its Classic Papers product is carried out at the level of broad subject categories, checking whether these documents are also covered in Web of Science and Scopus, and whether the citation counts offered by the different sources are similar. The results show that a large fraction of highly-cited documents in the Social Sciences and Humanities (8.6–28.2%) are invisible to Web of Science and Scopus. In the Natural, Life, and Health Sciences the proportion of missing highly-cited documents in Web of Science and Scopus is much lower. Furthermore, in all areas, Spearman correlation coefficients of citation counts in Google Scholar, as compared to Web of Science and Scopus citation counts, are remarkably strong (.83–.99). The main conclusion is that the data about highly-cited documents available in the inclusive database Google Scholar does indeed reveal significant coverage deficiencies in Web of Science and Scopus in several areas of research. Therefore, using these selective databases to compute bibliometric indicators based on counts of highly-cited documents might produce biased assessments in poorly covered areas.

Improvements in Google Scholar Citations are for the summer: creating an institutional affiliation link feature

The lost academic home: institutional affiliation links in Google Scholar Citations

Google Scholar: the 'big data' bibliographic tool

Google Scholar as a Data Source for Research Assessment

The Google scholar experiment: How to index false papers and manipulate bibliometric indicators

Can we use Google Scholar to identify highly-cited documents?

Manipulating Google Scholar Citations and Google Scholar Metrics: simple, easy and tempting

Google Scholar Metrics 2014: a low cost bibliometric tool

A novel method for depicting academic disciplines through Google Scholar Citations: The case of Bibliometrics

Google Scholar Metrics 2013: nothing new under the sun

The counting house: measuring those who count. Presence of Bibliometrics, Scientometrics, Informetrics, Webometrics and Altmetrics in the Google Scholar Citations, ResearcherID, ResearchGate, Mendeley & Twitter

The dark side of Open Access in Google and Google Scholar: the case of Latin-American repositories

Ranking journals: Could Google Scholar Metrics be an alternative to Journal Citation Reports and Scimago Journal Rank?

Google Scholar and the h-index in biomedicine: the popularization of bibliometric asessment

Google Scholar and the gray literature: A reply to Bonato's review

A New Era in Citation and Bibliometric Analyses: Web of Science, Scopus, and Google Scholar

Mining Google Scholar Citations: An Exploratory Study.

Google Scholar as a tool for discovering journal articles in library and information science

Back to the past: on the shoulders of an academic search engine giant

Coverage of highly-cited documents in Google Scholar, Web of Science, and Scopus: a multidisciplinary comparison

Measuring the Impact of Digital Heritage Collections Using Google Scholar