Abstract:Background: Systematic literature review (SLR), a robust method to identify and summarize evidence from published sources, is considered to be a complex, time-consuming, labor-intensive, and expensive task. Objective: This study aimed to present a solution based on natural language processing (NLP) that accelerates and streamlines the SLR process for observational studies using real-world data. Methods: We followed an agile software development and iterative software engineering methodology to build a customized intelligent end-to-end living NLP-assisted solution for observational SLR tasks. Multiple machine learning-based NLP algorithms were adopted to automate article screening and data element extraction processes. The NLP prediction results can be further reviewed and verified by domain experts, following the human-in-the-loop design. The system integrates explainable articificial intelligence to provide evidence for NLP algorithms and add transparency to extracted literature data elements. The system was developed based on 3 existing SLR projects of observational studies, including the epidemiology studies of human papillomavirus-associated diseases, the disease burden of pneumococcal diseases, and cost-effectiveness studies on pneumococcal vaccines. Results: Our Intelligent SLR Platform covers major SLR steps, including study protocol setting, literature retrieval, abstract screening, full-text screening, data element extraction from full-text articles, results summary, and data visualization. The NLP algorithms achieved accuracy scores of 0.86-0.90 on article screening tasks (framed as text classification tasks) and macroaverage F1 scores of 0.57-0.89 on data element extraction tasks (framed as named entity recognition tasks). Conclusions: Cutting-edge NLP algorithms expedite SLR for observational studies, thus allowing scientists to have more time to focus on the quality of data and the synthesis of evidence in observational studies. Aligning the living SLR concept, the system has the potential to update literature data and enable scientists to easily stay current with the literature related to observational studies prospectively and continuously.

Leveraging AI to improve evidence synthesis in conservation

ADVISE: AI-accelerated Design of Evidence Synthesis for Global Development

Leveraging Generative AI for Clinical Evidence Summarization Needs to Ensure Trustworthiness

Harnessing Artificial Intelligence for Wildlife Conservation

Accelerating Clinical Evidence Synthesis with Large Language Models

Leveraging artificial intelligence to enhance systematic reviews in health research: advanced tools and challenges

Tapping into non-English-language science for the conservation of global biodiversity

Accelerating Evidence Synthesis in Observational Studies: Development of a Living Natural Language Processing-Assisted Intelligent Systematic Literature Review System

Benchmarking Human-AI Collaboration for Common Evidence Appraisal Tools

Data extraction for evidence synthesis using a large language model: A proof‐of‐concept study

Semi-Automated evidence synthesis in health psychology: current methods and future prospects

The Convergence of AI and Synthetic Biology: The Looming Deluge

Using artificial intelligence methods for systematic review in health sciences: A systematic review

Building Understandable Messaging for Policy and Evidence Review (BUMPER) with AI

Unveiling the Larsen effect in the scientific literature domain: Navigating quality amidst the artificial intelligence‐driven deluge

Bayesian evidence synthesis as a flexible alternative to meta-analysis: A simulation study and empirical demonstration

A Review and Categorization of Artificial Intelligence-Based Opportunities in Wildlife, Ocean and Land Conservation

Elephants and Algorithms: A Review of the Current and Future Role of AI in Elephant Monitoring

Artificial intelligence to unlock real‐world evidence in clinical oncology: A primer on recent advances

Adapting to the Impact of AI in Scientific Writing: Balancing Benefits and Drawbacks while Developing Policies and Regulations

Evidence Is All You Need: Ordering Imaging Studies via Language Model Alignment with the ACR Appropriateness Criteria