Citation manipulation found to be rife in math

Michele Catanzaro

DOI: https://doi.org/10.1126/science.ado3859

IF: 56.9

2024-02-02

Science

Abstract:Problem is so severe that an influential top-researchers list has excluded the entire field

multidisciplinary sciences

What problem does this paper attempt to address?

Citations versus expert opinions: citation analysis of featured reviews of the American Mathematical Society

Lawrence Smolinsky,Daniel S. Sage,Aaron J. Lercher,Aaron Cao

DOI: https://doi.org/10.1007/s11192-021-03894-2

IF: 3.801

2021-03-02

Scientometrics

Abstract:Peer review and citation metrics are two means of gauging the value of scientific research, but the lack of publicly available peer review data makes the comparison of these methods difficult. Mathematics can serve as a useful laboratory for considering these questions because as an exact science, there is a narrow range of reasons for citations. In mathematics, virtually all published articles are post-publication reviewed by mathematicians in Mathematical Reviews (MathSciNet) and so the data set was essentially the Web of Science mathematics publications from 1993 to 2004. For a decade, especially important articles were singled out in Mathematical Reviews for featured reviews. In this study, we analyze the bibliometrics of elite articles selected by peer review and by citation count. We conclude that the two notions of significance described by being a featured review article and being highly cited are distinct. This indicates that peer review and citation counts give largely independent determinations of highly distinguished articles. We also consider whether hiring patterns of subfields and mathematicians' interest in subfields reflect subfields of featured review or highly cited articles. We re-examine data from two earlier studies in light of our methods for implications on the peer review/citation count relationship to a diversity of disciplines.

information science & library science,computer science, interdisciplinary applications
The Role of Positive and Negative Citations in Scientific Evaluation

Xiaomei Bai,Ivan Lee,Zhaolong Ning,Amr Tolba,Feng Xia

DOI: https://doi.org/10.48550/arXiv.2008.03844

2020-08-10

Abstract:Quantifying the impact of scientific papers objectively is crucial for research output assessment, which subsequently affects institution and country rankings, research funding allocations, academic recruitment and national/international scientific priorities. While most of the assessment schemes based on publication citations may potentially be manipulated through negative citations, in this study, we explore Conflict of Interest (COI) relationships and discover negative citations and subsequently weaken the associated citation strength. PANDORA (Positive And Negative COI- Distinguished Objective Rank Algorithm) has been developed, which captures the positive and negative COI, together with the positive and negative suspected COI relationships. In order to alleviate the influence caused by negative COI relationship, collaboration times, collaboration time span, citation times and citation time span are employed to determine the citing strength; while for positive COI relationship, we regard it as normal citation relationship. Furthermore, we calculate the impact of scholarly papers by PageRank and HITS algorithms, based on a credit allocation algorithm which is utilized to assess the impact of institutions fairly and objectively. Experiments are conducted on the publication dataset from American Physical Society (APS) dataset, and the results demonstrate that our method significantly outperforms the current solutions in Recommendation Intensity of list R at top-K and Spearman's rank correlation coefficient at top-K.

Digital Libraries,Information Retrieval
The Self-Cited Rate of Scientific Journals and the Manipulation of Their Impact Factors

Guang Yu,Liang Wang

DOI: https://doi.org/10.1007/s11192-007-1779-8

IF: 3.801

2007-01-01

Scientometrics

Abstract:Owing to some discussions about manipulating impact factor by requesting authors to increase their citations to the publication journal, we theoretically establish a mathematical expression of a relation between the journal self-citation rate and its impact factor by the single-factor method in this paper. Based on self-citation data of some journals in JCR and the observed relation between journal impact factor and the self-cited rate, we analyze the possibility that journal editors manipulate impact factors of their journals by raising the self-cited rate. Finally, we make some suggestions for supervising this crude way of active manipulating the impact factor.
Theory of citing

M.V. Simkin,V.P. Roychowdhury

DOI: https://doi.org/10.1007/978-1-4614-0754-6_16

2011-09-11

Abstract:We present empirical data on misprints in citations to twelve high-profile papers. The great majority of misprints are identical to misprints in articles that earlier cited the same paper. The distribution of the numbers of misprint repetitions follows a power law. We develop a stochastic model of the citation process, which explains these findings and shows that about 70-90% of scientific citations are copied from the lists of references used in other papers. Citation copying can explain not only why some misprints become popular, but also why some papers become highly cited. We show that a model where a scientist picks few random papers, cites them, and copies a fraction of their references accounts quantitatively for empirically observed distribution of citations.

Physics and Society,Disordered Systems and Neural Networks,Applications
‘Hidden’ citations conceal the true impact of research

Michael Allen

DOI: https://doi.org/10.1088/2058-7058/37/07/13

2024-07-01

Physics World

Abstract:Papers introducing concepts that have since become common knowledge are often under-cited by researchers, skewing those articles’ true impact.

physics, multidisciplinary
Citation sentence reuse behavior of scientists: A case study on massive bibliographic text dataset of computer science

Mayank Singh,Abhishek Niranjan,Divyansh Gupta,Nikhil Angad Bakshi,Animesh Mukherjee,Pawan Goyal

DOI: https://doi.org/10.48550/arXiv.1705.02499

2017-05-07

Abstract:Our current knowledge of scholarly plagiarism is largely based on the similarity between full text research articles. In this paper, we propose an innovative and novel conceptualization of scholarly plagiarism in the form of reuse of explicit citation sentences in scientific research articles. Note that while full-text plagiarism is an indicator of a gross-level behavior, copying of citation sentences is a more nuanced micro-scale phenomenon observed even for well-known researchers. The current work poses several interesting questions and attempts to answer them by empirically investigating a large bibliographic text dataset from computer science containing millions of lines of citation sentences. In particular, we report evidences of massive copying behavior. We also present several striking real examples throughout the paper to showcase widespread adoption of this undesirable practice. In contrast to the popular perception, we find that copying tendency increases as an author matures. The copying behavior is reported to exist in all fields of computer science; however, the theoretical fields indicate more copying than the applied fields.

Digital Libraries
Non-consistency, Non-Cited Items, and the Impact Factor: A Consequence of the Arithmetic

Ronald Rousseau,Loet Leydesdorff

2011-01-01

Abstract:We show that as a consequence of basic properties of elementary arithmetic journal impact factors show a counterintuitive behaviour with respect to adding non-cited articles. Synchronous as well as diachronous journal impact factors are affected. Our findings provide a rationale for not taking uncitable publications into account in impact factor calculations, at least if these items are truly uncitable.
Spanish rector accused of inflating citation count

Cathleen O'Grady

DOI: https://doi.org/10.1126/science.adt5689

IF: 56.9

2024-10-05

Science

Abstract:Dozens of Springer Nature papers flagged for excessively citing Juan Manuel Corchado

multidisciplinary sciences
Uncited articles and their effect on the concentration of citations

Diego Kozlowski1,Jens Peter Andersen,Vincent Larivière

DOI: https://doi.org/10.1002/asi.24852

2023-06-16

Abstract:Empirical evidence demonstrates that citations received by scholarly publications follow a pattern of preferential attachment, resulting in a power-law distribution. Such asymmetry has sparked significant debate regarding the use of citations for research evaluation. However, a consensus has yet to be established concerning the historical trends in citation concentration. Are citations becoming more concentrated in a small number of articles? Or have recent geopolitical and technical changes in science led to more decentralized distributions? This ongoing debate stems from a lack of technical clarity in measuring inequality. Given the variations in citation practices across disciplines and over time, it is crucial to account for multiple factors that can influence the findings. This article explores how reference-based and citation-based approaches, uncited articles, citation inflation, the expansion of bibliometric databases, disciplinary differences, and self-citations affect the evolution of citation concentration. Our results indicate a decreasing trend in citation concentration, primarily driven by a decline in uncited articles, which, in turn, can be attributed to the growing significance of Asia and Europe. On the whole, our findings clarify current debates on citation concentration and show that, contrary to a widely-held belief, citations are increasingly scattered.

Digital Libraries,Computers and Society,Physics and Society
The prestige and status of research fields within mathematics

Jean-Marc Schlenker

DOI: https://doi.org/10.48550/arXiv.2008.13244

2020-08-31

Abstract:While the ``hierarchy of science'' has been widely analysed, there is no corresponding study of the status of subfields within a given scientific field. We use bibliometric data to show that subfields of mathematics have a different ``standing'' within the mathematics community. Highly ranked departments tend to specialize in some subfields more than in others, and the same subfields are also over-represented in the most selective mathematics journals or among recipients of top prizes. Moreover this status of subfields evolves markedly over the period of observation (1984--2016), with some subfields gaining and others losing in standing. The status of subfields is related to different publishing habits, but some of those differences are opposite to those observed when considering the hierarchy of scientific fields. We examine possible explanations for the ``status'' of different subfields. Some natural explanations -- availability of funding, importance of applications -- do not appear to function, suggesting that factors internal to the discipline are at work. We propose a different type of explanation, based on a notion of ``focus'' of a subfield, that might or might not be specific to mathematics.

Digital Libraries,History and Overview
Distinct citation distributions complicate research evaluations. A single indicator that universally reveals research efficiency cannot be formulated

Alonso Rodríguez-Navarro

2024-07-12

Abstract:Purpose: Analyze the diversity of citation distributions to publications in different research topics to investigate the accuracy of size-independent, rank-based indicators. Top percentile-based indicators are the most common indicators of this type, and the evaluations of Japan are the most evident misjudgments. Design/methodology/approach: The distributions of citations to publications from countries and in journals in several research topics were analyzed along with the corresponding global publications using histograms with logarithmic binning, double rank plots, and normal probability plots of log-transformed numbers of citations. Findings: Size-independent, top percentile-based indicators are accurate when the global ranks of local publications fit a power law, but deviations in the least cited papers are frequent in countries and occur in all journals with high impact factors. In these cases, a single indicator is misleading. Comparisons of proportions of uncited papers are the best way to predict these deviations. Research limitations: The study is fundamentally analytical; its results describe mathematical facts that are self-evident. Practical implications: Respectable institutions, such as the OECD, European Commission, US National Science Board, and others, produce research country rankings and individual evaluations using size-independent percentile indicators that are misleading in many countries. These misleading evaluations should be discontinued because they cause confusion among research policymakers and lead to incorrect research policies. Originality/value: Studies linking the lower tail of citation distribution, including uncited papers, to percentile research indicators have not been performed previously. The present results demonstrate that studies of this type are necessary to find reliable procedures for research assessments.

Digital Libraries,Information Retrieval
On the share of mathematics published by Elsevier and Springer

Jean-Christophe Mourrat

DOI: https://doi.org/10.48550/arXiv.1905.05635

2019-05-13

Abstract:For-profit editors such as Elsevier and Springer have been subject to sustained criticism from academics and university libraries, including calls to boycott, and discontinued subscriptions. Mathematicians have played a particularly active role in this critique, and have endeavored to imagine new publication practices and create new journals. This motivates the monitoring of the share of articles published by different editors. I used data from MathSciNet over the period 2000-2017, and focused on the 100 journals with highest citations per article. Within this category, the share of articles published by Elsevier and Springer has steadily increased over this period, from about a third to almost half of the total.

Digital Libraries,History and Overview
Can We Mathematically Spot Possible Manipulation of Results in Research Manuscripts Using Benford's Law?

Teddy Lazebnik,Dan Gorlitsky

DOI: https://doi.org/10.3390/data8110165

2023-07-04

Abstract:The reproducibility of academic research has long been a persistent issue, contradicting one of the fundamental principles of science. What is even more concerning is the increasing number of false claims found in academic manuscripts recently, casting doubt on the validity of reported results. In this paper, we utilize an adaptive version of Benford's law, a statistical phenomenon that describes the distribution of leading digits in naturally occurring datasets, to identify potential manipulation of results in research manuscripts, solely using the aggregated data presented in those manuscripts. Our methodology applies the principles of Benford's law to commonly employed analyses in academic manuscripts, thus, reducing the need for the raw data itself. To validate our approach, we employed 100 open-source datasets and successfully predicted 79% of them accurately using our rules. Additionally, we analyzed 100 manuscripts published in the last two years across ten prominent economic journals, with ten manuscripts randomly sampled from each journal. Our analysis predicted a 3% occurrence of result manipulation with a 96% confidence level. Our findings uncover disturbing inconsistencies in recent studies and offer a semi-automatic method for their detection.

Information Retrieval,Digital Libraries
Citation inequity and gendered citation practices in contemporary physics

Erin G. Teich,Jason Z. Kim,Christopher W. Lynn,Samantha C. Simon,Andrei A. Klishin,Karol P. Szymula,Pragya Srivastava,Lee C. Bassett,Perry Zurn,Jordan D. Dworkin,Dani S. Bassett

DOI: https://doi.org/10.1038/s41567-022-01770-1

2021-12-17

Abstract:The historical and contemporary under-attribution of women's contributions to scientific scholarship is well-known and well-studied, with effects that are felt today in myriad ways by women scientists. One measure of this under-attribution is the so-called citation gap between men and women: the under-citation of papers authored by women relative to expected rates coupled with a corresponding over-citation of papers authored by men relative to expected rates. We explore the citation gap in contemporary physics, analyzing over one million articles published over the last 25 years in 35 physics journals that span a wide range of subfields. Using a model that predicts papers' expected citation rates according to a set of characteristics separate from author gender, we find a global bias wherein papers authored by women are significantly under-cited, and papers authored by men are significantly over-cited. Moreover, we find that citation behavior varies along several dimensions, such that imbalances differ according to who is citing, where they are citing, and how they are citing. Specifically, citation imbalance in favor of man-authored papers is highest for papers authored by men, papers published in general physics journals, and papers likely to be less familiar to citing authors. Our results suggest that, although deciding which papers to cite is an individual choice, the cumulative effects of these choices needlessly harm a subset of scholars. We discuss several strategies for the mitigation of these effects, including conscious behavioral changes at the individual, journal, and community levels.

Physics and Society,Digital Libraries
Discovering Mathematical Objects of Interest -- A Study of Mathematical Notations

Andre Greiner-Petter,Moritz Schubotz,Fabian Mueller,Corinna Breitinger,Howard S. Cohl,Akiko Aizawa,Bela Gipp

DOI: https://doi.org/10.1145/3366423.3380218

2021-06-22

Abstract:Mathematical notation, i.e., the writing system used to communicate concepts in mathematics, encodes valuable information for a variety of information search and retrieval systems. Yet, mathematical notations remain mostly unutilized by today's systems. In this paper, we present the first in-depth study on the distributions of mathematical notation in two large scientific corpora: the open access arXiv (2.5B mathematical objects) and the mathematical reviewing service for pure and applied mathematics zbMATH (61M mathematical objects). Our study lays a foundation for future research projects on mathematical information retrieval for large scientific corpora. Further, we demonstrate the relevance of our results to a variety of use-cases. For example, to assist semantic extraction systems, to improve scientific search engines, and to facilitate specialized math recommendation systems. The contributions of our presented research are as follows: (1) we present the first distributional analysis of mathematical formulae on arXiv and zbMATH; (2) we retrieve relevant mathematical objects for given textual search queries (e.g., linking $P_{n}^{(\alpha, \beta)}\!\left(x\right)$ with `Jacobi polynomial'); (3) we extend zbMATH's search engine by providing relevant mathematical formulae; and (4) we exemplify the applicability of the results by presenting auto-completion for math inputs as the first contribution to math recommendation systems. To expedite future research projects, we have made available our source code and data.

Digital Libraries,Information Retrieval
Citation bias, diversity, and ethics

Keisha S. Ray,Perry Zurn,Jordan D. Dworkin,Dani S. Bassett,David B. Resnik

DOI: https://doi.org/10.1080/08989621.2022.2111257

2022-08-20

Accountability in Research

Abstract:How often a researcher is cited usually plays a decisive role in that person's career advancement, because academic institutions often use citation metrics, either explicitly or implicitly, to estimate research impact and productivity. Research has shown, however, that citation patterns and practices are affected by various biases, including the prestige of the authors being cited and their gender, race, and nationality, whether self-attested or perceived. Some commentators have proposed that researchers can address biases related to social identity or position by including a Citation Diversity Statement in a manuscript submitted for publication. A Citation Diversity Statement is a paragraph placed before the reference section of a manuscript in which the authors address the diversity and equitability of their references in terms of gender, race, ethnicity, or other factors and affirm a commitment to promoting equity and diversity in sources and references. The present commentary considers arguments in favor of Citation Diversity Statements, and some practical and ethical issues that these statements raise.
Addressing citation manipulation: Advice for authors, reviewers and editors

Vadim A. Markel

DOI: https://doi.org/10.1016/j.rinp.2024.107618

IF: 4.565

2024-04-03

Results in Physics

Abstract:An inappropriate request for citations (IRC) occurs when a reviewer asks the authors of the reviewed manuscript to cite multiple papers with the sole purpose of boosting the reviewer's own citation index. In the past several years, we have witnessed a steady increase of IRC cases both in terms of frequency and scope. While some relevant ethical guidelines have been provided by the Committee on Publication Ethics (COPE), we feel that the problem is persistent and requires a coordinated response from editors, authors and reviewers. This article provides advice from the editors of Results in Physics for identifying and avoiding IRCs as well as for preventing the appearance of scientifically unmotivated citations in published papers.

physics, multidisciplinary,materials science
Citation Sentiment Reflects Multiscale Sociocultural Norms

Xiaohuan Xia,Mathieu Ouellet,Shubhankar P. Patankar,Diana I. Tamir,Dani S. Bassett

2024-12-06

Abstract:Modern science is formally structured around scholarly publication, where scientific knowledge is canonized through citation. Precisely how citations are given and accrued can provide information about the value of discovery, the history of scientific ideas, the structure of fields, and the space or scope of inquiry. Yet parsing this information has been challenging because citations are not simply present or absent; rather, they differ in purpose, function, and sentiment. In this paper, we investigate how critical and favorable sentiments are distributed across citations, and demonstrate that citation sentiment tracks sociocultural norms across scales of collaboration, discipline, and country. At the smallest scale of individuals, we find that researchers cite scholars they have collaborated with more favorably (and less critically) than scholars they have not collaborated with. Outside collaborative relationships, higher h-index scholars cite lower h-index scholars more critically. At the mesoscale of disciplines, we find that wetlab disciplines tend to be less critical than drylab disciplines, and disciplines that engage in more synthesis through publishing more review articles tend to be less critical. At the largest scale of countries, we find that greater individualism (and lesser acceptance of the unequal distribution of power) is associated with more critical sentiment. Collectively, our results demonstrate how sociocultural factors can explain variations in sentiment in scientific communication. As such, our study contributes to a broader understanding of how human factors influence the practice of science, and underscores the importance of considering the larger sociocultural contexts in which science progresses.

Social and Information Networks
Symbolic Manipulators Affect Mathematical Mindsets

Thomas J. Bing,Edward F. Redish

DOI: https://doi.org/10.1119/1.2835053

2007-12-08

Abstract:Symbolic calculators like Mathematica are becoming more commonplace among upper level physics students. The presence of such a powerful calculator can couple strongly to the type of mathematical reasoning students employ. It does not merely offer a convenient way to perform the computations students would have otherwise wanted to do by hand. This paper presents examples from the work of upper level physics majors where Mathematica plays an active role in focusing and sustaining their thought around calculation. These students still engage in powerful mathematical reasoning while they calculate but struggle because of the narrowed breadth of their thinking. Their reasoning is drawn into local attractors where they look to calculation schemes to resolve questions instead of, for example, mapping the mathematics to the physical system at hand. We model the influence of Mathematica as an integral part of the constant feedback that occurs in how students frame, and hence focus, their work.

Physics Education
Global citation inequality is on the rise

Mathias Wullum Nielsen,Jens Peter Andersen

DOI: https://doi.org/10.1073/pnas.2012208118

IF: 11.1

2021-02-08

Proceedings of the National Academy of Sciences

Abstract:Significance By analyzing a global sample of 4 million authors and 26 million scientific papers, this study finds that the top 1% most-cited scientists have increased their cumulative citation shares from 14 to 21% between 2000 and 2015 and that the Gini coefficient for citation imbalance has risen from 0.65 to 0.70. The growing citation concentration should be understood in the context of diverging trends in publication and collaboration activities for the top 1% compared to the “ordinary scientist.” Our study raises intriguing questions about how rising inequalities will shape the evolution of science.

Citation manipulation found to be rife in math

Citations versus expert opinions: citation analysis of featured reviews of the American Mathematical Society

The Role of Positive and Negative Citations in Scientific Evaluation

The Self-Cited Rate of Scientific Journals and the Manipulation of Their Impact Factors

Theory of citing

‘Hidden’ citations conceal the true impact of research

Citation sentence reuse behavior of scientists: A case study on massive bibliographic text dataset of computer science

Non-consistency, Non-Cited Items, and the Impact Factor: A Consequence of the Arithmetic

Spanish rector accused of inflating citation count

Uncited articles and their effect on the concentration of citations

The prestige and status of research fields within mathematics

Distinct citation distributions complicate research evaluations. A single indicator that universally reveals research efficiency cannot be formulated

On the share of mathematics published by Elsevier and Springer

Can We Mathematically Spot Possible Manipulation of Results in Research Manuscripts Using Benford's Law?

Citation inequity and gendered citation practices in contemporary physics

Discovering Mathematical Objects of Interest -- A Study of Mathematical Notations

Citation bias, diversity, and ethics

Addressing citation manipulation: Advice for authors, reviewers and editors

Citation Sentiment Reflects Multiscale Sociocultural Norms

Symbolic Manipulators Affect Mathematical Mindsets

Global citation inequality is on the rise