Abstract:Background: The study of disease progression relies on clinical data, including text data, and extracting valuable features from text data has been a research hot spot. With the rise of large language models (LLMs), semantic-based extraction pipelines are gaining acceptance in clinical research. However, the security and feature hallucination issues of LLMs require further attention. Objective: This study aimed to introduce a novel modular LLM pipeline, which could semantically extract features from textual patient admission records. Methods: The pipeline was designed to process a systematic succession of concept extraction, aggregation, question generation, corpus extraction, and question-and-answer scale extraction, which was tested via 2 low-parameter LLMs: Qwen-14B-Chat (QWEN) and Baichuan2-13B-Chat (BAICHUAN). A data set of 25,709 pregnancy cases from the People's Hospital of Guangxi Zhuang Autonomous Region, China, was used for evaluation with the help of a local expert's annotation. The pipeline was evaluated with the metrics of accuracy and precision, null ratio, and time consumption. Additionally, we evaluated its performance via a quantified version of Qwen-14B-Chat on a consumer-grade GPU. Results: The pipeline demonstrates a high level of precision in feature extraction, as evidenced by the accuracy and precision results of Qwen-14B-Chat (95.52% and 92.93%, respectively) and Baichuan2-13B-Chat (95.86% and 90.08%, respectively). Furthermore, the pipeline exhibited low null ratios and variable time consumption. The INT4-quantified version of QWEN delivered an enhanced performance with 97.28% accuracy and a 0% null ratio. Conclusions: The pipeline exhibited consistent performance across different LLMs and efficiently extracted clinical features from textual data. It also showed reliable performance on consumer-grade hardware. This approach offers a viable and effective solution for mining clinical research data from textual records.

Medical Record Text Analysis Based on Latent Semantic Analysis

A Method of Automatic Annotation for Medical Record Text Based on Latent Dirichlet Allocation

Medical Record Semantic Analysis Based on Weighted LDA

Cross-Domain Learning Based Traditional Chinese Medicine Medical Record Classification.

Traditional Chinese Medicine Clinical Records Classification Using Knowledge-Powered Document Embedding

Medical Language Processing Technology and Application

Medical Aided Diagnosis Using Electronic Medical Records Based On Lda And Word Vector Model

Hierarchical organization for medical video summarization using latent visual and semantic analysis

Automatic Conversion of Electronic Medical Record Text for OpenEHR Based on Semantic Analysis

Topic model based medical record classification method

Electronic Medical Data Analysis Based on Word Vector and Deep Learning Model

Lexical Characteristics Analysis of Chinese Clinical Documents

Semantic Analysis For Enhanced Medical Retrieval

Re-Structuring and Specific Similarity Computation of Electronic Medical Records

MaLDA:medication analysis based on LDA

An Entity Extraction Pipeline for Medical Text Records Using Large Language Models: Analytical Study

Using a bag of Words for Automatic Medical Image Annotation with a Latent Semantic

Exploiting Distributional Semantics to Benefit Machine Learning in Automated Classification of Chinese Clinical Text.

An Ensemble Semantic Textual Similarity Measure Based on Multiple Evidences for Biomedical Documents

An Improved Double Channel Long Short-Term Memory Model for Medical Text Classification

Black-Box Segmentation of Electronic Medical Records