Abstract:Objective: Breast cancer is the most common malignant tumor among women. The diagnosis and treatment information of breast cancer patients is abundant in multiple types of clinical fields, including clinicopathological data, genotype and phenotype information, treatment information, and prognosis information. However, current studies are mainly focused on extracting information from one specific type of clinical field. This study defines a comprehensive information model to represent the whole-course clinical information of patients. Furthermore, deep learning approaches are used to extract the concepts and their attributes from clinical breast cancer documents by fine-tuning pretrained Bidirectional Encoder Representations from Transformers (BERT) language models. Materials and methods: The clinical corpus that was used in this study was from one 3A cancer hospital in China, consisting of the encounter notes, operation records, pathology notes, radiology notes, progress notes and discharge summaries of 100 breast cancer patients. Our system consists of two components: a named entity recognition (NER) component and a relation recognition component. For each component, we implemented deep learning-based approaches by fine-tuning BERT, which outperformed other state-of-the-art methods on multiple natural language processing (NLP) tasks. A clinical language model is first pretrained using BERT on a large-scale unlabeled corpus of Chinese clinical text. For NER, the context embeddings that were pretrained using BERT were used as the input features of the Bi-LSTM-CRF (Bidirectional long-short-memory-conditional random fields) model and were fine-tuned using the annotated breast cancer notes. Furthermore, we proposed an approach to fine-tune BERT for relation extraction. It was considered to be a classification problem in which the two entities that were mentioned in the input sentence were replaced with their semantic types. Results: Our best-performing system achieved F1 scores of 93.53% for the NER and 96.73% for the relation extraction. Additional evaluations showed that the deep learning-based approaches that fine-tuned BERT did outperform the traditional Bi-LSTM-CRF and CRF machine learning algorithms in NER and the attention-Bi-LSTM and SVM (support vector machines) algorithms in relation recognition. Conclusion: In this study, we developed a deep learning approach that fine-tuned BERT to extract the breast cancer concepts and their attributes. It demonstrated its superior performance compared to traditional machine learning algorithms, thus supporting its uses in broader NER and relation extraction tasks in the medical domain.

ClinicalRadioBERT: Knowledge-Infused Few Shot Learning for Clinical Notes Named Entity Recognition.

ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission

Clinical-BERT: Vision-Language Pre-training for Radiograph Diagnosis and Reports Generation.

Information extraction from weakly structured radiological reports with natural language queries

Extracting comprehensive clinical information for breast cancer using deep learning methods

Ensemble Transfer Learning on Augmented Domain Resources for Oncological Named Entity Recognition in Chinese Clinical Records

Multifaceted Natural Language Processing Task–Based Evaluation of Bidirectional Encoder Representations From Transformers Models for Bilingual (Korean and English) Clinical Notes: Algorithm Development and Validation

Accurate Medical Named Entity Recognition Through Specialized NLP Models

Natural Language Processing Model for Identifying Critical Findings-A Multi-Institutional Study

Infusing clinical knowledge into tokenisers for language models

Exploring the performance and explainability of fine-tuned BERT models for neuroradiology protocol assignment

Evaluation of a BERT Natural Language Processing Model for Automating CT and MRI Triage and Protocol Selection

BERT Based Clinical Knowledge Extraction for Biomedical Knowledge Graph Construction and Analysis

Character-Based Deep Learning Approaches for Clinical Named Entity Recognition: A Comparative Study Using Chinese EHR Texts.

Information Extraction from Clinical Notes: Are We Ready to Switch to Large Language Models?

Language inference-based learning for Low-Resource Chinese clinical named entity recognition using language model

Use of BERT (Bidirectional Encoder Representations from Transformers)-Based Deep Learning Method for Extracting Evidences in Chinese Radiology Reports: Development of a Computer-Aided Liver Cancer Diagnosis Framework

An End-to-End Natural Language Processing System for Automatically Extracting Radiation Therapy Events From Clinical Texts

Application of a Domain-specific BERT for Detection of Speech Recognition Errors in Radiology Reports.

Lightweight transformers for clinical natural language processing

Highly accurate classification of chest radiographic reports using a deep learning natural language model pre-trained on 3.8 million text reports