More than 2 million research papers have disappeared from the Internet

Sarah Wild

DOI: https://doi.org/10.1038/d41586-024-00616-5

IF: 64.8

2024-03-05

Nature

Abstract:An analysis of DOIs suggests that digital preservation is not keeping up with burgeoning scholarly knowledge. An analysis of DOIs suggests that digital preservation is not keeping up with burgeoning scholarly knowledge.

multidisciplinary sciences

What problem does this paper attempt to address?

Open is not forever: A study of vanished open access journals

Mikael Laakso,Lisa Matthias,Najko Jahn

DOI: https://doi.org/10.1002/asi.24460

2021-02-21

Journal of the Association for Information Science and Technology

Abstract:<p>The preservation of the scholarly record has been a point of concern since the beginning of knowledge production. With print publications, the responsibility rested primarily with librarians, but the shift toward digital publishing and, in particular, the introduction of open access (OA) have caused ambiguity and complexity. Consequently, the long‐term accessibility of journals is not always guaranteed, and they can even disappear from the web completely. The focus of this exploratory study is on the phenomenon of vanished journals, something that has not been carried out before. For the analysis, we consulted several major bibliographic indexes, such as Scopus, Ulrichsweb, and the Directory of Open Access Journals, and traced the journals through the Internet Archive's Wayback Machine. We found 174 OA journals that, through lack of comprehensive and open archives, vanished from the web between 2000 and 2019, spanning all major research disciplines and geographic regions of the world. Our results raise vital concern for the integrity of the scholarly record and highlight the urgency to take collaborative action to ensure continued access and prevent the loss of more scholarly knowledge. We encourage those interested in the phenomenon of vanished journals to use the public dataset for their own research.</p>

information science & library science,computer science, information systems
Analysis of the deletions of DOIs: What factors undermine their persistence and to what extent?

Jiro Kikkawa,Masao Takaku,Fuyuki Yoshikane

DOI: https://doi.org/10.48550/arXiv.2207.12018

2022-07-25

Digital Libraries

Abstract:Digital Object Identifiers (DOIs) are regarded as persistent; however, they are sometimes deleted. Deleted DOIs are an important issue not only for persistent access to scholarly content but also for bibliometrics, because they may cause problems in correctly identifying scholarly articles. However, little is known about how much of deleted DOIs and what causes them. We identified deleted DOIs by comparing the datasets of all Crossref DOIs on two different dates, investigated the number of deleted DOIs in the scholarly content along with the corresponding document types, and analyzed the factors that cause deleted DOIs. Using the proposed method, 708,282 deleted DOIs were identified. The majority corresponded to individual scholarly articles such as journal articles, proceedings articles, and book chapters. There were cases of many DOIs assigned to the same content, e.g., retracted journal articles and abstracts of international conferences. We show the publishers and academic societies which are the most common in deleted DOIs. In addition, the top cases of single scholarly content with a large number of deleted DOIs were revealed. The findings of this study are useful for citation analysis and altmetrics, as well as for avoiding deleted DOIs.
Daily briefing: Millions of online papers at risk of disappearing

Flora Graham

DOI: https://doi.org/10.1038/d41586-024-00664-x

IF: 64.8

2024-03-04

Nature

Abstract:A big chunk of the literature is missing from online archives, an anti-methane satellite will seek leaks and how to weigh a neutrino.

multidisciplinary sciences
The availability of research data declines rapidly with article age

Timothy Vines,Arianne Albert,Rose Andrew,Florence Debarré,Dan Bock,Michelle Franklin,Kimberley Gilbert,Jean-Sébastien Moore,Sébastien Renaut,Diana J. Rennison

DOI: https://doi.org/10.1016/j.cub.2013.11.014

2013-12-20

Abstract:Policies ensuring that research data are available on public archives are increasingly being implemented at the government [1], funding agency [2-4], and journal [5,6] level. These policies are predicated on the idea that authors are poor stewards of their data, particularly over the long term [7], and indeed many studies have found that authors are often unable or unwilling to share their data [8-11]. However, there are no systematic estimates of how the availability of research data changes with time since publication. We therefore requested datasets from a relatively homogenous set of 516 articles published between 2 and 22 years ago, and found that availability of the data was strongly affected by article age. For papers where the authors gave the status of their data, the odds of a dataset being extant fell by 17% per year. In addition, the odds that we could find a working email address for the first, last or corresponding author fell by 7% per year. Our results reinforce the notion that, in the long term, research data cannot be reliably preserved by individual researchers, and further demonstrate the urgent need for policies mandating data sharing via public archives.

Digital Libraries,Physics and Society,Populations and Evolution
Disappearing repositories -- taking an infrastructure perspective on the long-term availability of research data

Dorothea Strecker,Heinz Pampel,Rouven Schabinger,Nina Leonie Weisweiler

DOI: https://doi.org/10.48550/arXiv.2310.06712

2023-10-10

Abstract:Currently, there is limited research investigating the phenomenon of research data repositories being shut down, and the impact this has on the long-term availability of data. This paper takes an infrastructure perspective on the preservation of research data by using a registry to identify 191 research data repositories that have been closed and presenting information on the shutdown process. The results show that 6.2 % of research data repositories indexed in the registry were shut down. The risks resulting in repository shutdown are varied. The median age of a repository when shutting down is 12 years. Strategies to prevent data loss at the infrastructure level are pursued to varying extent. 44 % of the repositories in the sample migrated data to another repository, and 12 % maintain limited access to their data collection. However, both strategies are not permanent solutions. Finally, the general lack of information on repository shutdown events as well as the effect on the findability of data and the permanence of the scholarly record are discussed.

Digital Libraries
Measuring data rot: An analysis of the continued availability of shared data from a Single University

Kristin A. Briney

DOI: https://doi.org/10.1371/journal.pone.0304781

IF: 3.7

2024-06-06

PLoS ONE

Abstract:To determine where data is shared and what data is no longer available, this study analyzed data shared by researchers at a single university. 2166 supplemental data links were harvested from the university's institutional repository and web scraped using R. All links that failed to scrape or could not be tested algorithmically were tested for availability by hand. Trends in data availability by link type, age of publication, and data source were examined for patterns. Results show that researchers shared data in hundreds of places. About two-thirds of links to shared data were in the form of URLs and one-third were DOIs, with several FTP links and links directly to files. A surprising 13.4% of shared URL links pointed to a website homepage rather than a specific record on a website. After testing, 5.4% the 2166 supplemental data links were found to be no longer available. DOIs were the type of shared link that was least likely to disappear with a 1.7% loss, with URL loss at 5.9% averaged over time. Links from older publications were more likely to be unavailable, with a data disappearance rate estimated at 2.6% per year, as well as links to data hosted on journal websites. The results support best practice guidance to share data in a data repository using a permanent identifier.

multidisciplinary sciences
Persistent URIs Must Be Used To Be Persistent

Herbert Van de Sompel,Martin Klein,Shawn M. Jones

DOI: https://doi.org/10.48550/arXiv.1602.09102

2016-03-01

Abstract:We quantify the extent to which references to papers in scholarly literature use persistent HTTP URIs that leverage the Digital Object Identifier infrastructure. We find a significant number of references that do not, speculate why authors would use brittle URIs when persistent ones are available, and propose an approach to alleviate the problem.

Digital Libraries
Losing My Revolution: How Many Resources Shared on Social Media Have Been Lost?

Hany M. SalahEldeen,Michael L. Nelson

DOI: https://doi.org/10.48550/arXiv.1209.3026

2012-09-13

Digital Libraries

Abstract:Social media content has grown exponentially in the recent years and the role of social media has evolved from just narrating life events to actually shaping them. In this paper we explore how many resources shared in social media are still available on the live web or in public web archives. By analyzing six different event-centric datasets of resources shared in social media in the period from June 2009 to March 2012, we found about 11% lost and 20% archived after just a year and an average of 27% lost and 41% archived after two and a half years. Furthermore, we found a nearly linear relationship between time of sharing of the resource and the percentage lost, with a slightly less linear relationship between time of sharing and archiving coverage of the resource. From this model we conclude that after the first year of publishing, nearly 11% of shared resources will be lost and after that we will continue to lose 0.02% per day.
Global visibility of publications through Digital Object Identifiers

Houcemeddine Turki,Grischa Fraumann,Mohamed Ali Hadj Taieb,Mohamed Ben Aouicha

DOI: https://doi.org/10.3389/frma.2023.1207980

2023-08-21

Abstract:This brief research report analyzes the availability of Digital Object Identifiers (DOIs) worldwide, highlighting the dominance of large publishing houses and the need for unique persistent identifiers to increase the visibility of publications from developing countries. The study reveals that a considerable amount of publications from developing countries are excluded from the global flow of scientific information due to the absence of DOIs, emphasizing the need for alternative publishing models. The authors suggest that the availability of DOIs should receive more attention in scholarly communication and scientometrics, contributing to a necessary debate on DOIs relevant for librarians, publishers, and scientometricians.

Digital Libraries,Social and Information Networks
Analysis of Sci-Hub downloads of computer science papers

Darko Andročec

DOI: https://doi.org/10.1515/ausi-2017-0006

2017-07-26

Abstract:Abstract The scientific knowledge is disseminated by research papers. Most of the research literature is copyrighted by publishers and avail- able only through paywalls. Recently, some websites offer most of the recent content for free. One of them is the controversial website Sci-Hub that enables access to more than 47 million pirated research papers. In April 2016, Science Magazine published an article on Sci-Hub activity over the period of six months and publicly released the Sci-Hub’s server log data. The mentioned paper aggregates the view that relies on all downloads and for all fields of study, but these findings might be hiding interesting patterns within computer science. The mentioned Sci-Hub log data was used in this paper to analyse downloads of computer science papers based on DBLP’s list of computer science publications. The top downloads of computer science papers were analysed, together with the geographical location of Sci-Hub users, the most downloaded publishers, types of papers downloaded, and downloads of computer science papers per publication year. The results of this research can be used to improve legal access to the most relevant scientific repositories or journals for the computer science field.
Four decades of counterfeit research: A bibliometric analysis

Irfan Butt,Maha Khamis Al Balushi,Seung Hwan Lee,Myuri Mohan,Naseer Ahmad Khan,Shelley Haines,Seung Hwan (Mark) Lee

DOI: https://doi.org/10.1080/23311975.2023.2284814

2023-11-27

Cogent Business & Management

Abstract:This paper assesses the evolution of last 43 years in counterfeit research with respect to sources of knowledge (i.e. journals, authors, institutions, countries) and research themes. The oldest paper on this subject discovered in the Scopus database was published 43 years ago, yet a time frame was not specified. Sources of knowledge are assessed on research productivity (quantitative) as well as impact (qualitative). Research themes, key areas of focus within the counterfeit research landscape, are identified and discussed to conceptualize our understanding of the field. Via a systematic literature review, 713 peer-reviewed academic articles published in 282 journals from 1978 to 2021 were selected as the sample for this study. The systematic review technique was chosen as compared with narrative reviews of the literature it focuses on open, extensive, and detailed approaches to literature searches, in addition to conforming to the scientific criteria utilised in primary research, namely transparency, rigour, comprehensiveness, and reproducibility. A database of references and citations was created for analysis. The data was analyzed to prepare comparative tables. Further, the Leximancer software was used to generate lexical conceptual trends. This data was further analyzed to identify emerging themes. The Journal of Business Ethics had the highest number of articles and citations, followed by the Journal of Business Research and Business Horizons. Ian Phau (14 articles) and Michael D. Smith, (9 articles) were the most prolific authors. Joseph Nunes and Ian Phau attained the highest number of citations, cited 658 and 577 times respectively. Eight major research themes were identified: products, piracy, model, price, firms, digital, supply, and ethical. Each theme was analyzed over time. The major research areas analyzed across the articles over time were Technology (particularly “Technology” and “Software” topics) and Ethics (particularly “IP” and “Legislation”). The identification of these research area captures the essence of the paper’s uniqueness and contribution to this field of research. This is the first systematic literature review in counterfeit literature that captures multi-decade sources of knowledge in business journals.
Inconsistency in the registration of the Digital Object Identifier (DOI) of articles on Web of Science and Scopus

Erwin Krauskopf,Mauricio Salgado

DOI: https://doi.org/10.22201/iibi.24488321xe.2023.96.58784

2023-08-26

Abstract:Almost 25 years ago, the Digital Object Identifier (DOI) was implemented with the purpose of providing a unique and persistent form of document identification. As DOIs can be assigned to any object, journals rapidly adopted their use as it eases the process of identifying a specific document, thus increasing its visibility. However, while studying the impact of papers published by Chilean researchers in disciplines related to Social Sciences, Arts, and Humanities, we noticed that some journals published a considerable proportion of documents without DOIs. Thus, we questioned if this was due to a lack of DOI adoption by these journals or if it was a database processing error. Our findings indicate that while most journals have adopted the use of DOI for article-type documents, many of the Web of Science and Scopus records do not include this information. To overcome this issue, databases need to ensure the accuracy and consistency of their data, while the editorial management team of each journal ought to confirm that all the metadata from their articles has been properly registered by these databases. Undoubtedly, the use of DOI benefits all academics as it facilitates the discoverability and retrieval of the published articles.
Cited But Not Archived: Analyzing the Status of Code References in Scholarly Articles

Emily Escamilla,Martin Klein,Talya Cooper,Vicky Rampin,Michele C. Weigle,Michael L. Nelson

2024-01-10

Abstract:One in five arXiv articles published in 2021 contained a URI to a Git Hosting Platform (GHP), which demonstrates the growing prevalence of GHP URIs in scholarly publications. However, GHP URIs are vulnerable to the same reference rot that plagues the Web at large. The disappearance of software hosting platforms, like Gitorious and Google Code, and the source code they contain threatens research reproducibility. Archiving the source code and development history available in GHPs enables the long-term reproducibility of research. Software Heritage and Web archives contain archives of GHP URI resources. However, are the GHP URIs referenced by scholarly publications contained within the Software Heritage and Web archive collections? We analyzed a dataset of GHP URIs extracted from scholarly publications to determine (1) is the URI still publicly available on the live Web?, (2) has the URI been archived by Software Heritage?, and (3) has the URI been archived by Web archives? Of all GHP URIs, we found that 93.98% were still publicly available on the live Web, 68.39% had been archived by Software Heritage, and 81.43% had been archived by Web archives.

Digital Libraries
An analysis of retracted papers in Computer Science

Martin Shepperd,Leila Yousefi

DOI: https://doi.org/10.1371/journal.pone.0285383

2022-06-14

Abstract:Context: The retraction of research papers, for whatever reason, is a growing phenomenon. However, although retracted paper information is publicly available via publishers, it is somewhat distributed and inconsistent. Objective: The aim is to assess: (i) the extent and nature of retracted research in Computer Science (CS) (ii) the post-retraction citation behaviour of retracted works and (iii) the potential impact on systematic reviews and mapping studies. Method: We analyse the Retraction Watch database and take citation information from the Web of Science and Google scholar. Results: We find that of the 33,955 entries in the Retraction watch database (16 May 2022), 2,816 are classified as CS, i.e., approximately 8.3%. For CS, 56% of retracted papers, provide little or no information as to the reasons. This contrasts with 26% for other disciplines. There is also a remarkable disparity between different publishers, a tendency for multiple versions of a retracted paper over and above the Version of Record (VoR), and for new citations long after a paper is officially retracted. Conclusions: Unfortunately retraction seems to be a sufficiently common outcome for a scientific paper that we as a research community need to take it more seriously, e.g., standardising procedures and taxonomies across publishers and the provision of appropriate research tools. Finally, we recommend particular caution when undertaking secondary analyses and meta-analyses which are at risk of becoming contaminated by these problem primary studies.

Information Retrieval
A Quantitative Analysis of Long-term Preservation of Digital Resources

Zhang Jing,Shao Bo

DOI: https://doi.org/10.3969/j.issn.1672-514X.2011.01.012

2011-01-01

Abstract:Taking the CNKI Database as the source origin and the theme as the retrieval method,and using the quantitative methods,this paper aims to confirm the core periodicals,core authors and core institution and studythe current situation,existing problems and countermeasure,by analyzing the long-term preservation of digital resources in China in the ten years 1999—2008 from several aspects,such as the time and periodicals distribution,authors,institution,fund assistance and themes of papers.
Digital Infrastructures for Scholarly Content Objects

Jodi Schneider,Anita de Waard,Wolf-Tilo Balke,Xiaoguang Wang,Ningyuan Song,Bolin Hua,Yuanxi Fu

DOI: https://doi.org/10.1109/jcdl52503.2021.00069

2021-01-01

Abstract:As digital libraries make the dissemination of research publications easier, they also enable the propagation of invalid or unreliable knowledge. Examples of relevant problems include: retraction and inadvertent citation and reuse of retracted papers [1], [2]; propagation of errors in literature and scientific databases [3], [4]; non-reproducible papers; known domain-specific issues such as cell line contamination [5]; bias in research datasets and publications [6]–[8]; systematic reviews that arrive at different conclusions about the same question at the same time [9], [10]. The digital environment facilitates broad interdisciplinary reuse beyond the originating scientific community; thus, marking known problems and tracing the impact on dependent and follow-on works is particularly important (but still under-addressed). Further, context-specific information inside a paper may not be immediately reusable when extracted by automated processes, leading to apparent contradictions [11]. Current mitigating approaches use the underlying reasoning for information retrieval [12], [13], develop new infrastructures analyzing the reasoning [14]–[16] or certainty [17] of statements, or use visualization to highlight possible discrepancies [10], [15].
Exclusive: the papers that most heavily cite retracted studies

Richard Van Noorden,Miryam Naddaf

DOI: https://doi.org/10.1038/d41586-024-02719-5

IF: 64.8

2024-08-31

Nature

Abstract:Data from giant project show how withdrawn research propagates through the literature. Data from giant project show how withdrawn research propagates through the literature.

multidisciplinary sciences
Loss of sustainability in scientific work

Niklas Reisz,Vito D.P. Servedio,Vittorio Loreto,William Schueller,Márcia R. Ferreira,Stefan Thurner

DOI: https://doi.org/10.1088/1367-2630/ac6ca1

2022-02-03

Abstract:For decades the number of scientific publications has been rapidly increasing, effectively out-dating knowledge at a tremendous rate. Only few scientific milestones remain relevant and continuously attract citations. Here we quantify how long scientific work remains being utilized, how long it takes before today's work is forgotten, and how milestone papers differ from those forgotten. To answer these questions, we study the complete temporal citation network of all American Physical Society journals. We quantify the probability of attracting citations for individual publications based on age and the number of citations they have received in the past. We capture both aspects, the forgetting and the tendency to cite already popular works, in a microscopic generative model for the dynamics of scientific citation networks. We find that the probability of citing a specific paper declines with age as a power law with an exponent of $\alpha \sim -1.4$. Whenever a paper in its early years can be characterized by a scaling exponent above a critical value, $\alpha_c$, the paper is likely to become "ever-lasting". We validate the model with out-of-sample predictions, with an accuracy of up to 90% (AUC $\sim 0.9$). The model also allows us to estimate an expected citation landscape of the future, predicting that 95% of papers cited in 2050 have yet to be published. The exponential growth of articles, combined with a power-law type of forgetting and papers receiving fewer and fewer citations on average, suggests a worrying tendency toward information overload and raises concerns about scientific publishing's long-term sustainability.

Physics and Society
‘Hijacked’ journals mar top database

Jeffrey Brainard

DOI: https://doi.org/10.1126/science.adn3529

IF: 56.9

2023-12-08

Science

Abstract:Scopus listed hundreds of papers published on stolen sitesScopus listed hundreds of papers published on stolen sites

multidisciplinary sciences
Patterns of retractions from 1981-2020 : Does a fraud lead to another fraud?

Kiran Sharma

DOI: https://doi.org/10.48550/arXiv.2011.13091

2020-11-26

Digital Libraries

Abstract:Misconduct accounts for the majority of retracted scientific publications and this database reveals the disturbing trend in science~\citep{fang2012misconduct, brainard2018massive}. The objective of the study is to find the association among the authors' collaboration, the number of retracted papers, the number of retracted citations, journal impact factor, and research areas. We present a detailed analysis of 12231 research papers indexed by Web of Science (WoS) as retracted publications from 1981-2020. The study demonstrates the collaboration patterns of retracted publications where 61.5% of authors have only one and 24.6% have two retracted papers; however, 2% of authors have more than 10 retracted papers. To study the impact of citing retracted papers, we investigated the retracted papers with citations. The study reveals that 55.2% of retracted papers have been cited at least once, where 25.4% of papers are such papers where at least one citation turned out to be a retraction. This shows the impact of scientific misconduct or fraud on new research. The number of retractions is independent of the journal impact factor and as compared to high impact papers, low impact papers are attracting more citations. We also investigate the citations received by retracted papers published in higher as well as lower impact factor journals. 1/4th of the papers are retracted citations that cited the retracted papers; however, there is no significant relationship exists between the higher impact or lower impact journals with retractions or citations. Finally, how the average team size and average retracted citations vary among different research areas are studied. The study provides an insight that how a fraud leads to another fraud in the scientific world. Also, the rising trend of citations of retracted papers is a serious concern.

More than 2 million research papers have disappeared from the Internet

Open is not forever: A study of vanished open access journals

Analysis of the deletions of DOIs: What factors undermine their persistence and to what extent?

Daily briefing: Millions of online papers at risk of disappearing

The availability of research data declines rapidly with article age

Disappearing repositories -- taking an infrastructure perspective on the long-term availability of research data

Measuring data rot: An analysis of the continued availability of shared data from a Single University

Persistent URIs Must Be Used To Be Persistent

Losing My Revolution: How Many Resources Shared on Social Media Have Been Lost?

Global visibility of publications through Digital Object Identifiers

Analysis of Sci-Hub downloads of computer science papers

Four decades of counterfeit research: A bibliometric analysis

Inconsistency in the registration of the Digital Object Identifier (DOI) of articles on Web of Science and Scopus

Cited But Not Archived: Analyzing the Status of Code References in Scholarly Articles

An analysis of retracted papers in Computer Science

A Quantitative Analysis of Long-term Preservation of Digital Resources

Digital Infrastructures for Scholarly Content Objects

Exclusive: the papers that most heavily cite retracted studies

Loss of sustainability in scientific work

‘Hijacked’ journals mar top database

Patterns of retractions from 1981-2020 : Does a fraud lead to another fraud?