Abstract:<h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Background</h3><p>Finding specific scientific articles in a large collection is an important natural language processing challenge in the biomedical domain. Systematic reviews and interactive article search are the type of downstream applications that benefit from addressing this problem. The task often involves screening articles for a combination of selection criteria. While machine learning was previously used for this purpose, it is not known if different criteria should be modeled together or separately in an ensemble model. The performance impact of the modern contextual language models on the task is also not known.</p><h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Methods</h3><p>We framed the problem as text classification and conducted experiments to compare ensemble architectures, where the selection criteria were mapped to the components of the ensemble. We proposed a novel cascade ensemble analogous to the step-wise screening process employed in developing the gold standard. We compared performance of the ensembles with a single integrated model, which we refer to as the individual task learner (ITL). We used SciBERT, a variant of BERT pre-trained on scientific articles, and conducted experiments using a manually annotated dataset of ∼49K MEDLINE abstracts, known as Clinical Hedges.</p><h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Results</h3><p>The cascade ensemble had significantly higher precision (0.663 vs. 0.388 vs. 0.478 vs. 0.320) and F measure (0.753 vs. 0.553 vs. 0.628 vs. 0.477) than ITL and ensembles using Boolean logic and a feed-forward network. However, ITL had significantly higher recall than the other classifiers (0.965 vs. 0.872 vs. 0.917 vs. 0.944). In fixed high recall studies, ITL achieved 0.509 precision @ 0.970 recall and 0.381 precision @ 0.985 recall on a subset that was studied earlier, and 0.295 precision @ 0.985 recall on the full dataset, all of which were improvements over the previous studies.</p><h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Conclusion</h3><p>Pre-trained neural contextual language models (e.g. SciBERT) performed well for screening scientific articles. Performance at high fixed recall makes the single integrated model (ITL) more suitable among the architectures considered here, for systematic reviews. However, high F measure of the cascade ensemble makes it a better approach for interactive search applications. The effectiveness of the cascade ensemble architecture suggests broader applicability beyond this task and the dataset, and the approach is analogous to query optimization in Information Retrieval and query optimization in databases.</p>

Cited text span identification for scientific summarisation using pre-trained encoders

T-BERTSum: Topic-Aware Text Summarization Based on BERT

A Supervised Approach to Extractive Summarisation of Scientific Papers

Assessment of Transformer-Based Encoder-Decoder Model for Human-Like Summarization

Data-driven Summarization of Scientific Articles

Enhancing Scientific Papers Summarization with Citation Graph

Improving Biomedical Abstractive Summarisation with Knowledge Aggregation from Citation Papers

Identification of paraphrased text in research articles through improved embeddings and fine-tuned BERT model

Fine-tuning Pre-trained Contextual Embeddings for Citation Content Analysis in Scholarly Publication

SKT5SciSumm -- Revisiting Extractive-Generative Approach for Multi-Document Scientific Summarization

Focus-Based Text Summarisation with Hybrid Embeddings

Using the contextual language model BERT for multi-criteria classification of scientific articles

Contextual embedded text summarizer system: A hybrid approach

Synthesizing Scientific Summaries: An Extractive and Abstractive Approach

Empowering Interdisciplinary Research with BERT-Based Models: An Approach Through SciBERT-CNN with Topic Modeling

SciSummPip: An Unsupervised Scientific Paper Summarization Pipeline

An Effective Contextual Language Modeling Framework for Speech Summarization with Augmented Features

Resolving Citation Links With Neural Networks

Semantic maps and metrics for science Semantic maps and metrics for science using deep transformer encoders

Improved hybrid text summarization system using deep contextualized embeddings and statistical features