For long-term sustainable software in bioinformatics
Luis Pedro Coelho
DOI: https://doi.org/10.1371/journal.pcbi.1011920
2024-03-16
PLoS Computational Biology
Abstract:Research software has become essential in science, including in the life sciences [1,2]. Most of this software is designed and built by research groups funded by short-term research grants. The coding is performed by trainees who move on to other jobs once the project is completed. This can result in dead software tools, whereby a tool described in a manuscript no longer works after publication [3,4]. Commercial software uses sales to pay for long-term maintenance and support. In addition to the financial costs, commercial software often lags behind the capabilities of academic software—a natural consequence of the fact that new ideas are often explored first in academia—and commercial software may not be open source, which poses a barrier for some researchers as it can hinder reproducibility. It is noteworthy, though, that, in the field of machine learning, it is not uncommon for commercial entities to release high-quality tools, including AlphaFold2 [5,6], as well as libraries such as PyTorch [7], which then are incorporated by academic research labs into their computational biology projects. During my training and now, as a group leader, I have created several open source research software tools. My group publicly commits to maintaining these for a minimum of 5 years after the publication of the associated manuscript. In practice, several of our projects have been maintained for longer given that they continue to be used. Ideally, software should be maintained for as long as it is being actively used. We also provide some basic support to users, in the form of addressing their queries. In this perspective, I will first define exactly what is meant by maintenance and when we are committing to it, then describe how we make it possible, and finally end with a call for the community and institutions to be explicit about what users can expect from each piece of research software. Maintaining software is different from extending it . Although the exact boundary between the 2 activities is not always clear cut; maintenance, to me, means that the software should perform as described in the manuscript. This means fixing any bugs that are reported. This may mean that changes are necessary, for example, to accommodate new versions of dependencies which break backwards compatibility (a common cause of software collapse [3]). If the software is provided as a service (e.g., as a web server), concerns related to security and privacy may lead to required updates and even lower-level network functionality may need to be updated to keep the service operational. All of the practices I will describe make it easier to extend the code to perform beyond what is described in the manuscript, but we do not commit to this. I also wish to distinguish between the code that we make available as Tools for others and code that we make available to support papers whose main purpose is to present biological findings. Both types of code play an important role in research and both should be made available. Given their different purposes, however, they are subject to different tradeoffs (see Box 1) and researchers should make it clear what is the purpose of the code they publish. In this context, there is extensive literature on reproducibility of computational results [8,9] and recommendations overlapping with our practices [10,11]. Most often, the focus has been on what I term Extended Methods Code (Level 1; see Box 1 ). However, this is a different challenge from the one that I address here, as I focus on code that implements Tools (Level 2; see Box 1 ). In fact, I consider that conflating these 2 classes of code deliverables (both of which are common in bioinformatics and computational biology) has hampered progress towards higher standards. Internally, in my group, I have found it very helpful to explicitly establish when there is an expectation that effort will be put into the code to make it widely applicable (a Tool ) and when it is tolerable that the code be less robust (although, naturally, still correct in its limited domain). Nonetheless, there are linkages between the reproducibility of analysis code and tool maintenance: on the one hand, reproducible research methods are the baseline for our work; on the other hand, tools no longer working is one of the reasons that results stop being reproducible [12,13] so that long-term tool maintenance will contribute to diminishing the reproducibility problem. These distinctions were developed internally in my group, but they mirror the first 3 levels of Hong and colleagues [14]. The more complex levels they consider are those that go beyond the small projects considered here and include critical tools such as BLAST [15] or STRING [16]. For those projects, with millions of users and thousands of yearly citations, the strategies I present here will -Abstract Truncated-
biochemical research methods,mathematical & computational biology