Abstract:The increasing expansion of biomedical documents has increased the number of natural language textual resources related to the current applications. Meanwhile, there has been a great interest in extracting useful information from meaningful coherent groupings of textual content documents in the last decade. However, it is challenging to discover informative representations and define relevant articles from the rapidly growing biomedical literature due to the unsupervised nature of document clustering. Moreover, empirical investigations demonstrated that traditional text clustering methods produce unsatisfactory results in terms of non-contextualized vector space representations because that neglect the semantic relationship between biomedical texts. Recently, pre-trained language models have emerged as successful in a wide range of natural language processing applications. In this paper, we propose the Gaussian Mixture Model-based efficient clustering framework that incorporates substantially pre-trained (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining) BioBERT domain-specific language representations to enhance the clustering accuracy. Our proposed framework consists of main three phases. First, classic text pre-processing techniques are used biomedical document data, which crawled from the PubMed repository. Second, representative vectors are extracted from a pre-trained BioBERT language model for biomedical text mining. Third, we employ the Gaussian Mixture Model as a clustering algorithm, which allows us to assign labels for each biomedical document. In order to prove the efficiency of our proposed model, we conducted a comprehensive experimental analysis utilizing several clustering algorithms while combining diverse embedding techniques. Consequently, the experimental results show that the proposed model outperforms the benchmark models by reaching performance measures of Fowlkes mallows score, silhouette coefficient, adjusted rand index, Davies-Bouldin score of 0.7817, 0.3765, 0.4478, 1.6849, respectively. We expect the outcomes of this study will assist domain specialists in comprehending thematically cohesive documents in the healthcare field.

A Clustering Algorithm Based on Document Embedding to Identify Clinical Note Templates

Document Clustering Using Locality Preserving Indexing

Traditional Chinese Medicine Clinical Records Classification Using Knowledge-Powered Document Embedding

Clinical Documents Clustering Based on Medication/Symptom Names Using Multi-View Nonnegative Matrix Factorization

Profiling Clinical Researchers Effectively using Embeddings and Clustering

A knowledge-driven approach to biomedical document conceptualization

Mining Clinical Phrases from Nursing Notes to Discover Risk Factors of Patient Deterioration

Discovering Thematically Coherent Biomedical Documents Using Contextualized Bidirectional Encoder Representations from Transformers-Based Clustering

A Hybrid Strategy for Clustering Data Mining Documents.

A Document Ensemble Clustering Approach Via Dimensionality Reduction

Adaptive encoding-based evolutionary approach for Chinese document clustering

Enhancing Medline Document Clustering by Incorporating Mesh Semantic Similarity

From Free Text to Clusters of Content in Health Records: An Unsupervised Graph Partitioning Approach

Clustering clinical models from local electronic health records based on semantic similarity

Embedding, Aligning and Reconstructing Clinical Notes to Explore Sepsis

Ensemble Non-negative Matrix Factorization for Clustering Biomedical Documents

Extracting information from free text through unsupervised graph-based clustering: an application to patient incident records

Semi-Supervised Patient Similarity Clustering Algorithm Based on Electronic Medical Records

A Graph-Based Biomedical Literature Clustering Approach Utilizing Term's Global and Local Importance Information

Design and simulation of a document clustering algorithm based on genetic algorithm

Efficient Semisupervised MEDLINE Document Clustering with MeSH-Semantic and Global-Content Constraints