Artificial intelligence to automate the systematic review of scientific literature

José de la Torre-López,Aurora Ramírez,José Raúl Romero

DOI: https://doi.org/10.1007/s00607-023-01181-x

2024-01-14

Abstract:Artificial intelligence (AI) has acquired notorious relevance in modern computing as it effectively solves complex tasks traditionally done by humans. AI provides methods to represent and infer knowledge, efficiently manipulate texts and learn from vast amount of data. These characteristics are applicable in many activities that human find laborious or repetitive, as is the case of the analysis of scientific literature. Manually preparing and writing a systematic literature review (SLR) takes considerable time and effort, since it requires planning a strategy, conducting the literature search and analysis, and reporting the findings. Depending on the area under study, the number of papers retrieved can be of hundreds or thousands, meaning that filtering those relevant ones and extracting the key information becomes a costly and error-prone process. However, some of the involved tasks are repetitive and, therefore, subject to automation by means of AI. In this paper, we present a survey of AI techniques proposed in the last 15 years to help researchers conduct systematic analyses of scientific literature. We describe the tasks currently supported, the types of algorithms applied, and available tools proposed in 34 primary studies. This survey also provides a historical perspective of the evolution of the field and the role that humans can play in an increasingly automated SLR process.

Information Retrieval,Artificial Intelligence

What problem does this paper attempt to address?

This paper primarily explores the use of artificial intelligence (AI) to automate the process of Systematic Literature Review (SLR). SLR is a method in scientific research that involves in-depth analysis of relevant literature, which usually requires a significant amount of time and effort. With the development of AI technology, automation of certain tasks in SLR, such as searching, filtering, and extracting key information, has become possible, leading to reduced errors and improved efficiency.

The paper proposes three research questions:
1. Which stages of SLR have already been automated using AI?
2. What AI technologies support the automation of SLR tasks?
3. To what extent are humans involved in AI-driven SLR automation?

The authors conducted a systematic literature search and analyzed 34 major studies over the past 15 years to understand the application of AI in SLR automation. This analysis included the tasks, types of algorithms, and tools used, as well as the role of humans in this process. The study found that although AI can save time and resources, its current application is still limited, requiring human perspectives to complement its shortcomings.

The paper also outlines the three stages of SLR: planning (defining research questions and strategies), execution (searching and screening relevant literature), and reporting (writing the review). The application of AI in the execution stage has attracted particular attention, especially machine learning (ML) and natural language processing (NLP) techniques used for automatically selecting relevant papers.

In summary, this paper aims to summarize the latest developments in AI for automating SLR and emphasizes the open questions and challenges that need to be considered in future research, including unsupported tasks, potential AI technologies, and experimental reproducibility.