Abstract:Objective: Breast cancer is the most common malignant tumor among women. The diagnosis and treatment information of breast cancer patients is abundant in multiple types of clinical fields, including clinicopathological data, genotype and phenotype information, treatment information, and prognosis information. However, current studies are mainly focused on extracting information from one specific type of clinical field. This study defines a comprehensive information model to represent the whole-course clinical information of patients. Furthermore, deep learning approaches are used to extract the concepts and their attributes from clinical breast cancer documents by fine-tuning pretrained Bidirectional Encoder Representations from Transformers (BERT) language models. Materials and methods: The clinical corpus that was used in this study was from one 3A cancer hospital in China, consisting of the encounter notes, operation records, pathology notes, radiology notes, progress notes and discharge summaries of 100 breast cancer patients. Our system consists of two components: a named entity recognition (NER) component and a relation recognition component. For each component, we implemented deep learning-based approaches by fine-tuning BERT, which outperformed other state-of-the-art methods on multiple natural language processing (NLP) tasks. A clinical language model is first pretrained using BERT on a large-scale unlabeled corpus of Chinese clinical text. For NER, the context embeddings that were pretrained using BERT were used as the input features of the Bi-LSTM-CRF (Bidirectional long-short-memory-conditional random fields) model and were fine-tuned using the annotated breast cancer notes. Furthermore, we proposed an approach to fine-tune BERT for relation extraction. It was considered to be a classification problem in which the two entities that were mentioned in the input sentence were replaced with their semantic types. Results: Our best-performing system achieved F1 scores of 93.53% for the NER and 96.73% for the relation extraction. Additional evaluations showed that the deep learning-based approaches that fine-tuned BERT did outperform the traditional Bi-LSTM-CRF and CRF machine learning algorithms in NER and the attention-Bi-LSTM and SVM (support vector machines) algorithms in relation recognition. Conclusion: In this study, we developed a deep learning approach that fine-tuned BERT to extract the breast cancer concepts and their attributes. It demonstrated its superior performance compared to traditional machine learning algorithms, thus supporting its uses in broader NER and relation extraction tasks in the medical domain.

Pre-trained models, data augmentation, and ensemble learning for biomedical information extraction and document classification

Ensemble pretrained language models to extract biomedical knowledge from literature

Advancing Chinese biomedical text mining with community challenges

Community challenges in biomedical text mining over 10 years: success, failure and the future

BioBERT: a pre-trained biomedical language representation model for biomedical text mining

The biomedical relationship corpus of the BioRED track at the BioCreative VIII challenge and workshop

Optimized biomedical entity relation extraction method with data augmentation and classification using GPT-4 and Gemini

Text Mining Drug/Chemical-Protein Interactions using an Ensemble of BERT and T5 Based Models

Uncovering protein interaction in abstracts and text using a novel linear model and word proximity networks

AI for Biomedicine in the Era of Large Language Models

Automatic Text Summarization of COVID-19 Medical Research Articles using BERT and GPT-2

Abstract 5366: Constructing the largest-scale biomedical knowledge graph using all PubMed articles and its application in automated knowledge discovery

BioMamba: A Pre-trained Biomedical Language Representation Model Leveraging Mamba

The overview of the BioRED (Biomedical Relation Extraction Dataset) track at BioCreative VIII

Extracting comprehensive clinical information for breast cancer using deep learning methods

Detecting experimental techniques and selecting relevant documents for protein-protein interactions from biomedical literature

CU-UD: text-mining drug and chemical-protein interactions with ensembles of BERT-based models

Integrating deep learning architectures for enhanced biomedical relation extraction: a pipeline approach

NeuroSym-BioCAT: Leveraging Neuro-Symbolic Methods for Biomedical Scholarly Document Categorization and Question Answering