The citation black market: schemes selling fake references alarm scientists

Dalmeet Singh Chawla

DOI: https://doi.org/10.1038/d41586-024-01672-7

IF: 64.8

2024-08-22

Nature

Abstract:The ways in which researchers can artificially inflate their reference counts are growing. The ways in which researchers can artificially inflate their reference counts are growing.

multidisciplinary sciences

What problem does this paper attempt to address?

Google Scholar is manipulatable

Hazem Ibrahim,Fengyuan Liu,Yasir Zaki,Talal Rahwan

2024-02-07

Abstract:Citations are widely considered in scientists' evaluation. As such, scientists may be incentivized to inflate their citation counts. While previous literature has examined self-citations and citation cartels, it remains unclear whether scientists can purchase citations. Here, we compile a dataset of ~1.6 million profiles on Google Scholar to examine instances of citation fraud on the platform. We survey faculty at highly-ranked universities, and confirm that Google Scholar is widely used when evaluating scientists. Intrigued by a citation-boosting service that we unravelled during our investigation, we contacted the service while undercover as a fictional author, and managed to purchase 50 citations. These findings provide conclusive evidence that citations can be bought in bulk, and highlight the need to look beyond citation counts.

Computational Engineering, Finance, and Science,Digital Libraries,Social and Information Networks,Physics and Society
Recovering uncaptured citations in a scholarly network: A two-step citation analysis to estimate publication importance

zhuoren jiang,xiaozhong liu,yan chen

DOI: https://doi.org/10.1002/asi.23475

2016-01-01

Abstract:AbstractThe citation relationships between publications, which are significant for assessing the importance of scholarly components within a network, have been used for various scientific applications. Missing citation metadata in scholarly databases, however, create problems for classical citation-based ranking algorithms and challenge the performance of citation-based retrieval systems. In this research, we utilize a two-step citation analysis method to investigate the importance of publications for which citation information is partially missing. First, we calculate the importance of the author and then use his importance to estimate the publication importance for some selected articles. To evaluate this method, we designed a simulation experiment-"random citation-missing"-to test the two-step citation analysis that we carried out with the Association for Computing Machinery ACM Digital Library DL. In this experiment, we simulated different scenarios in a large-scale scientific digital library, from high-quality citation data, to very poor quality data, The results show that a two-step citation analysis can effectively uncover the importance of publications in different situations. More importantly, we found that the optimized impact from the importance of an author first step is exponentially increased when the quality of citation decreases. The findings from this study can further enhance citation-based publication-ranking algorithms for real-world applications.
Citation sentence reuse behavior of scientists: A case study on massive bibliographic text dataset of computer science

Mayank Singh,Abhishek Niranjan,Divyansh Gupta,Nikhil Angad Bakshi,Animesh Mukherjee,Pawan Goyal

DOI: https://doi.org/10.48550/arXiv.1705.02499

2017-05-07

Abstract:Our current knowledge of scholarly plagiarism is largely based on the similarity between full text research articles. In this paper, we propose an innovative and novel conceptualization of scholarly plagiarism in the form of reuse of explicit citation sentences in scientific research articles. Note that while full-text plagiarism is an indicator of a gross-level behavior, copying of citation sentences is a more nuanced micro-scale phenomenon observed even for well-known researchers. The current work poses several interesting questions and attempts to answer them by empirically investigating a large bibliographic text dataset from computer science containing millions of lines of citation sentences. In particular, we report evidences of massive copying behavior. We also present several striking real examples throughout the paper to showcase widespread adoption of this undesirable practice. In contrast to the popular perception, we find that copying tendency increases as an author matures. The copying behavior is reported to exist in all fields of computer science; however, the theoretical fields indicate more copying than the applied fields.

Digital Libraries
Hidden Citations Obscure True Impact in Science

Xiangyi Meng,Onur Varol,Albert-László Barabási

2024-05-12

Abstract:References, the mechanism scientists rely on to signal previous knowledge, lately have turned into widely used and misused measures of scientific impact. Yet, when a discovery becomes common knowledge, citations suffer from obliteration by incorporation. This leads to the concept of hidden citation, representing a clear textual credit to a discovery without a reference to the publication embodying it. Here, we rely on unsupervised interpretable machine learning applied to the full text of each paper to systematically identify hidden citations. We find that for influential discoveries hidden citations outnumber citation counts, emerging regardless of publishing venue and discipline. We show that the prevalence of hidden citations is not driven by citation counts, but rather by the degree of the discourse on the topic within the text of the manuscripts, indicating that the more discussed is a discovery, the less visible it is to standard bibliometric analysis. Hidden citations indicate that bibliometric measures offer a limited perspective on quantifying the true impact of a discovery, raising the need to extract knowledge from the full text of the scientific corpus.

Computation and Language,Digital Libraries,Social and Information Networks,Physics and Society
Fake research papers flagged by analysing authorship trends

Dalmeet Singh Chawla

DOI: https://doi.org/10.1038/d41586-024-00344-w

IF: 64.8

2024-02-08

Nature

Abstract:A new approach to detecting fraudulent paper-mill studies focuses on patterns of co-authors rather than manuscript text. A new approach to detecting fraudulent paper-mill studies focuses on patterns of co-authors rather than manuscript text.

multidisciplinary sciences
Sneaked references: Cooked reference metadata inflate citation counts

Lonni Besançon,Guillaume Cabanac,Cyril Labbé,Alexander Magazinov

DOI: https://doi.org/10.1002/asi.24896

2023-10-04

Abstract:We report evidence of an undocumented method to manipulate citation counts involving 'sneaked' references. Sneaked references are registered as metadata for scientific articles in which they do not appear. This manipulation exploits trusted relationships between various actors: publishers, the Crossref metadata registration agency, digital libraries, and bibliometric platforms. By collecting metadata from various sources, we show that extra undue references are actually sneaked in at Digital Object Identifier (DOI) registration time, resulting in artificially inflated citation counts. As a case study, focusing on three journals from a given publisher, we identified at least 9% sneaked references (5,978/65,836) mainly benefiting two authors. Despite not existing in the articles, these sneaked references exist in metadata registries and inappropriately propagate to bibliometric dashboards. Furthermore, we discovered 'lost' references: the studied bibliometric platform failed to index at least 56% (36,939/65,836) of the references listed in the HTML version of the publications. The extent of the sneaked and lost references in the global literature remains unknown and requires further investigations. Bibliometric platforms producing citation counts should identify, quantify, and correct these flaws to provide accurate data to their patrons and prevent further citation gaming.

Digital Libraries
Four decades of counterfeit research: A bibliometric analysis

Irfan Butt,Maha Khamis Al Balushi,Seung Hwan Lee,Myuri Mohan,Naseer Ahmad Khan,Shelley Haines,Seung Hwan (Mark) Lee

DOI: https://doi.org/10.1080/23311975.2023.2284814

2023-11-27

Cogent Business & Management

Abstract:This paper assesses the evolution of last 43 years in counterfeit research with respect to sources of knowledge (i.e. journals, authors, institutions, countries) and research themes. The oldest paper on this subject discovered in the Scopus database was published 43 years ago, yet a time frame was not specified. Sources of knowledge are assessed on research productivity (quantitative) as well as impact (qualitative). Research themes, key areas of focus within the counterfeit research landscape, are identified and discussed to conceptualize our understanding of the field. Via a systematic literature review, 713 peer-reviewed academic articles published in 282 journals from 1978 to 2021 were selected as the sample for this study. The systematic review technique was chosen as compared with narrative reviews of the literature it focuses on open, extensive, and detailed approaches to literature searches, in addition to conforming to the scientific criteria utilised in primary research, namely transparency, rigour, comprehensiveness, and reproducibility. A database of references and citations was created for analysis. The data was analyzed to prepare comparative tables. Further, the Leximancer software was used to generate lexical conceptual trends. This data was further analyzed to identify emerging themes. The Journal of Business Ethics had the highest number of articles and citations, followed by the Journal of Business Research and Business Horizons. Ian Phau (14 articles) and Michael D. Smith, (9 articles) were the most prolific authors. Joseph Nunes and Ian Phau attained the highest number of citations, cited 658 and 577 times respectively. Eight major research themes were identified: products, piracy, model, price, firms, digital, supply, and ethical. Each theme was analyzed over time. The major research areas analyzed across the articles over time were Technology (particularly “Technology” and “Software” topics) and Ethics (particularly “IP” and “Legislation”). The identification of these research area captures the essence of the paper’s uniqueness and contribution to this field of research. This is the first systematic literature review in counterfeit literature that captures multi-decade sources of knowledge in business journals.
Self-citations in around a dozen countries are unusually high

Dalmeet Singh Chawla

DOI: https://doi.org/10.1038/d41586-024-00090-z

IF: 64.8

2024-01-11

Nature

Abstract:Researchers behind the analysis think that policy incentives in these places are to blame. Researchers behind the analysis think that policy incentives in these places are to blame.

multidisciplinary sciences
Retweet Us, We Will Retweet You: Spotting Collusive Retweeters Involved in Blackmarket Services

Hridoy Sankar Dutta,Aditya Chetan,Brihi Joshi,Tanmoy Chakraborty

DOI: https://doi.org/10.48550/arXiv.1806.08979

2018-06-23

Abstract:Twitter has increasingly become a popular platform to share news and user opinion. A tweet is considered to be important if it receives high number of affirmative reactions from other Twitter users via Retweets. Retweet count is thus considered as a surrogate measure for positive crowd-sourced reactions - high number of retweets of a tweet aid in making its topic trending. This in turn bolsters the social reputation of the author of the tweet. Since social reputation/impact of users/tweets influences many decisions (such as promoting brands, advertisement etc.), several blackmarket syndicates have actively been engaged in producing fake retweets in a collusive manner. Users who want to boost the impact of their tweets approach the blackmarket services, and gain retweets for their own tweets by either paying money (Premium Services) or by retweeting other customers' tweets. Thus they become customers of blackmarket syndicates and engage in fake activities. Interestingly, these customers are neither bots, nor even fake users - they are usually normal human beings; they express a mix of organic and inorganic retweeting activities, and there is no synchronicity across their behaviors. In this paper, we make a first attempt to investigate such blackmarket customers engaged in producing fake retweets. We collected and annotated a novel dataset comprising of customers of many blackmarket services and show how their social behavior differs from genuine users. We then use state-of-the-art supervised models to detect three types of customers (bots, promotional, normal) and genuine users. We achieve a Macro F1-score of 0.87 with SVM, outperforming four other baselines significantly. We further design a browser extension, SCoRe which, given the link of a tweet, spots its fake retweeters in real-time. We also collected users' feedback on the performance of SCoRe and obtained 85% accuracy.

Social and Information Networks
Duplicate and fake publications in the scientific literature: how many SCIgen papers in computer science?

Cyril Labbé,Dominique Labbé

DOI: https://doi.org/10.1007/s11192-012-0781-y

IF: 3.801

2012-06-22

Scientometrics

Abstract:Two kinds of bibliographic tools are used to retrieve scientific publications and make them available online. For one kind, access is free as they store information made publicly available online. For the other kind, access fees are required as they are compiled on information provided by the major publishers of scientific literature. The former can easily be interfered with, but it is generally assumed that the latter guarantee the integrity of the data they sell. Unfortunately, duplicate and fake publications are appearing in scientific conferences and, as a result, in the bibliographic services. We demonstrate a software method of detecting these duplicate and fake publications. Both the free services (such as Google Scholar and DBLP) and the charged-for services (such as IEEE Xplore) accept and index these publications.

information science & library science,computer science, interdisciplinary applications
The Google scholar experiment: How to index false papers and manipulate bibliometric indicators

Emilio Delgado López‐Cózar,Nicolás Robinson‐García,Daniel Torres‐Salinas,Emilio Delgado López-Cózar,Nicolás Robinson-García,Daniel Torres-Salinas

DOI: https://doi.org/10.1002/asi.23056

2013-11-11

Journal of the Association for Information Science and Technology

Abstract:Google Scholar has been well received by the research community. Its promises of free, universal, and easy access to scientific literature coupled with the perception that it covers the social sciences and the humanities better than other traditional multidisciplinary databases have contributed to the quick expansion of Google Scholar Citations and Google Scholar Metrics: 2 new bibliometric products that offer citation data at the individual level and at journal level. In this article, we show the results of an experiment undertaken to analyze Google Scholar's capacity to detect citation‐counting manipulation. For this, we uploaded 6 documents to an institutional web domain that were authored by a fictitious researcher and referenced all the publications of the members of the EC3 research group at the University of Granada. The detection by Google Scholar of these papers caused an outburst in the number of citations included in the Google Scholar Citations profiles of the authors. We discuss the effects of such an outburst and how it could affect the future development of such products, at both the individual level and the journal level, especially if Google Scholar persists with its lack of transparency.

information science & library science,computer science, information systems
The right to refuse unwanted citations: rethinking the culture of science around the citation

Jaime A. Teixeira da Silva,Quan-Hoang Vuong

DOI: https://doi.org/10.1007/s11192-021-03960-9

IF: 3.801

2021-05-08

Scientometrics

Abstract:Logically, and by most common standards, academics would be pleased to be cited, considering it a form of recognition of their intellect. In return, especially those with high citation counts, such as Clarivate Analytics' Highly Cited Researchers, can benefit through peer recognition, rewards, funding, securing a better position, or expanding a collaborative network. Despite known and untold benefits, one issue has not been discussed: the right to refuse to be cited or the right to refuse a citation. Academics might not want to be cited by papers published in truly predatory journals, papers with false authors, or sting papers with falsified elements that employ underhanded ethical tactics. Currently, academics generally have the freedom to select where they publish their findings and choose studies they cite, so it is highly probable that requests to remove citations or refuse citations might never become formal publishing policy. Nonetheless, this academic discussion is worth having as valid and invalid literature increasingly gets mixed through citations, and as the grey zone between predatory/non-predatory and scholarly/unscholarly becomes increasingly difficult to distinguish.

information science & library science,computer science, interdisciplinary applications
Uncited articles and their effect on the concentration of citations

Diego Kozlowski1,Jens Peter Andersen,Vincent Larivière

DOI: https://doi.org/10.1002/asi.24852

2023-06-16

Abstract:Empirical evidence demonstrates that citations received by scholarly publications follow a pattern of preferential attachment, resulting in a power-law distribution. Such asymmetry has sparked significant debate regarding the use of citations for research evaluation. However, a consensus has yet to be established concerning the historical trends in citation concentration. Are citations becoming more concentrated in a small number of articles? Or have recent geopolitical and technical changes in science led to more decentralized distributions? This ongoing debate stems from a lack of technical clarity in measuring inequality. Given the variations in citation practices across disciplines and over time, it is crucial to account for multiple factors that can influence the findings. This article explores how reference-based and citation-based approaches, uncited articles, citation inflation, the expansion of bibliometric databases, disciplinary differences, and self-citations affect the evolution of citation concentration. Our results indicate a decreasing trend in citation concentration, primarily driven by a decline in uncited articles, which, in turn, can be attributed to the growing significance of Asia and Europe. On the whole, our findings clarify current debates on citation concentration and show that, contrary to a widely-held belief, citations are increasingly scattered.

Digital Libraries,Computers and Society,Physics and Society
Mitigating Consequences of Prestige in Citations of Publications

Michael Balzer,Adhen Benlahlou

2024-11-08

Abstract:For many public research organizations, funding creation of science and maximizing scientific output is of central interest. Typically, when evaluating scientific production for funding, citations are utilized as a proxy, although these are severely influenced by factors beyond scientific impact. This study aims to mitigate the consequences of the Matthew effect in citations, where prominent authors and prestigious journals receive more citations regardless of the scientific content of the publications. To this end, the study presents an approach to predicting citations of papers based solely on observable characteristics available at the submission stage of a double-blind peer-review process. Combining classical linear models, generalized linear models and utilizing large-scale data sets on biomedical papers based on the PubMed database, the results demonstrate that it is possible to make fairly accurate predictions of citations using only observable characteristics of papers excluding information on authors and journals, thereby mitigating the Matthew effect. Thus, the outcomes have important implications for the field of scientometrics, providing a more objective method for citation prediction by relying on pre-publication variables that are immune to manipulation by authors and journals, thereby enhancing the objectivity of the evaluation process. Our approach is thus important for government agencies responsible for funding the creation of high-quality scientific content rather than perpetuating prestige.

Digital Libraries,Applications
‘Hidden’ citations conceal the true impact of research

Michael Allen

DOI: https://doi.org/10.1088/2058-7058/37/07/13

2024-07-01

Physics World

Abstract:Papers introducing concepts that have since become common knowledge are often under-cited by researchers, skewing those articles’ true impact.

physics, multidisciplinary
Manipulating Google Scholar Citations and Google Scholar Metrics: simple, easy and tempting

Emilio Delgado Lopez-Cozar,Nicolas Robinson-Garcia,Daniel Torres-Salinas

DOI: https://doi.org/10.48550/arXiv.1212.0638

2012-12-04

Digital Libraries

Abstract:The launch of Google Scholar Citations and Google Scholar Metrics may provoke a revolution in the research evaluation field as it places within every researchers reach tools that allow bibliometric measuring. In order to alert the research community over how easily one can manipulate the data and bibliometric indicators offered by Google s products we present an experiment in which we manipulate the Google Citations profiles of a research group through the creation of false documents that cite their documents, and consequently, the journals in which they have published modifying their H index. For this purpose we created six documents authored by a faked author and we uploaded them to a researcher s personal website under the University of Granadas domain. The result of the experiment meant an increase of 774 citations in 129 papers (six citations per paper) increasing the authors and journals H index. We analyse the malicious effect this type of practices can cause to Google Scholar Citations and Google Scholar Metrics. Finally, we conclude with several deliberations over the effects these malpractices may have and the lack of control tools these tools offer
Innovation or imitation: The diffusion of citations.

Chao Min,Ying Ding,Jiang Li,Yi Bu,Lei Pei,Jianjun Sun

DOI: https://doi.org/10.1002/asi.24047

2018-01-01

Journal of the Association for Information Science and Technology

Abstract:Citations in scientific literature are important both for tracking the historical development of scientific ideas and for forecasting research trends. However, the diffusion mechanisms underlying the citation process remain poorly understood, despite the frequent and longstanding use of citation counts for assessment purposes within the scientific community. Here, we extend the study of citation dynamics to a more general diffusion process to understand how citation growth associates with different diffusion patterns. Using a classic diffusion model, we quantify and illustrate specific diffusion mechanisms which have been proven to exert a significant impact on the growth and decay of citation counts. Experiments reveal a positive relation between the low p and low q pattern and high scientific impact. A sharp citation peak produced by rapid change of citation counts, however, has a negative effect on future impact. In addition, we have suggested a simple indicator, saturation level, to roughly estimate an individual article's current stage in the life cycle and its potential to attract future attention. The proposed approach can also be extended to higher levels of aggregation (e.g., individual scientists, journals, institutions), providing further insights into the practice of scientific evaluation.
“Lucky Points” of Scientists’ Referencing Behavior

Jialin Liu,Hongkan Chen,Zhibo Liu,Yi Bu

DOI: https://doi.org/10.2139/ssrn.4247238

2022-01-01

SSRN Electronic Journal

Abstract:In our recent study (Liu et al., 2022), we dissected a reference list with five distinct dimensions, namely the number of references, the number of citations of references, the age of references, the number of nodes in a reference cascade, and the density of bibliographic coupling networks.
Unveiling the ethical void: Bias in reference citations and its academic ramifications

Alicia Mattiazzi,Martin Vila-Petroff

DOI: https://doi.org/10.1016/j.crphys.2024.100130

2024-07-26

Abstract:Citation bias receives scant attention in discussions of ethics. However, inaccurate citation may lead to significant distortions in scientific understanding. Although ethnical and gender citation disparities have been proposed as critical aspects, there are other contributors to citation distortions, like region-based citation bias, that, although less recognized within the scientific community, are equally important. While the foundations of scientific citation include acknowledging pioneers, giving credit to related work, and providing background reading, other more subjective or even questionable criteria are often used when constructing a reference lists. Here, we discuss the potential causes and ethical concerns of citation bias, emphasizing the role of international- or region-based citation bias as one of the most harmful aspects of this ethical breach. We argue that the international scientific community should be aware of this problem and recognize its consequences, which include hindering the accurate dissemination of science, marginalizing underrepresented voices in academia, and impeding scientific progress. We advocate that scientists should compile their reference lists with the same seriousness and integrity they apply to all other aspects of their research.
Non-Linearity Between Referencing Behavior and Citation Impact: A Large-Scale, Discipline-Level Analysis

Jialin Liu,Hongkan Chen,Zhibo Liu,Yi Bu,Weiye Gu

DOI: https://doi.org/10.1016/j.joi.2022.101318

IF: 3.7

2022-01-01

Journal of Informetrics

Abstract:References formulate the foundation and endorsements of scientific research. Using articles pub-lished in 2005 covered by Microsoft Academic Graph, the current paper defines and calculates five indicators of references, i.e., the number of references, the number of citations of references, the age of references, the number of nodes in a reference cascade (a multi-generation reference network), and the density of bibliographic coupling networks by references, and investigates their relations with the citation impact of the focal publication. A non-linear relationship is shown in all the five indicators; specifically, we observe two types of patterns, namely inverted-L and-U relations, both presenting the existence of critical points. We further explore the discipline-level differences of such a relationship and how it relates to the characteristics of the discipline itself. Among all five indicators, the effect of disciplinary academic "environments " is universally iden-tified. We believe that the current paper provides insightful views to the discussion regarding the significance of a reference list.

The citation black market: schemes selling fake references alarm scientists

Google Scholar is manipulatable

Recovering uncaptured citations in a scholarly network: A two-step citation analysis to estimate publication importance

Citation sentence reuse behavior of scientists: A case study on massive bibliographic text dataset of computer science

Hidden Citations Obscure True Impact in Science

Fake research papers flagged by analysing authorship trends

Sneaked references: Cooked reference metadata inflate citation counts

Four decades of counterfeit research: A bibliometric analysis

Self-citations in around a dozen countries are unusually high

Retweet Us, We Will Retweet You: Spotting Collusive Retweeters Involved in Blackmarket Services

Duplicate and fake publications in the scientific literature: how many SCIgen papers in computer science?

The Google scholar experiment: How to index false papers and manipulate bibliometric indicators

The right to refuse unwanted citations: rethinking the culture of science around the citation

Uncited articles and their effect on the concentration of citations

Mitigating Consequences of Prestige in Citations of Publications

‘Hidden’ citations conceal the true impact of research

Manipulating Google Scholar Citations and Google Scholar Metrics: simple, easy and tempting

Innovation or imitation: The diffusion of citations.

“Lucky Points” of Scientists’ Referencing Behavior

Unveiling the ethical void: Bias in reference citations and its academic ramifications

Non-Linearity Between Referencing Behavior and Citation Impact: A Large-Scale, Discipline-Level Analysis