Abstract:The increasing expansion of biomedical documents has increased the number of natural language textual resources related to the current applications. Meanwhile, there has been a great interest in extracting useful information from meaningful coherent groupings of textual content documents in the last decade. However, it is challenging to discover informative representations and define relevant articles from the rapidly growing biomedical literature due to the unsupervised nature of document clustering. Moreover, empirical investigations demonstrated that traditional text clustering methods produce unsatisfactory results in terms of non-contextualized vector space representations because that neglect the semantic relationship between biomedical texts. Recently, pre-trained language models have emerged as successful in a wide range of natural language processing applications. In this paper, we propose the Gaussian Mixture Model-based efficient clustering framework that incorporates substantially pre-trained (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining) BioBERT domain-specific language representations to enhance the clustering accuracy. Our proposed framework consists of main three phases. First, classic text pre-processing techniques are used biomedical document data, which crawled from the PubMed repository. Second, representative vectors are extracted from a pre-trained BioBERT language model for biomedical text mining. Third, we employ the Gaussian Mixture Model as a clustering algorithm, which allows us to assign labels for each biomedical document. In order to prove the efficiency of our proposed model, we conducted a comprehensive experimental analysis utilizing several clustering algorithms while combining diverse embedding techniques. Consequently, the experimental results show that the proposed model outperforms the benchmark models by reaching performance measures of Fowlkes mallows score, silhouette coefficient, adjusted rand index, Davies-Bouldin score of 0.7817, 0.3765, 0.4478, 1.6849, respectively. We expect the outcomes of this study will assist domain specialists in comprehending thematically cohesive documents in the healthcare field.

A Text Document Clustering Method Based on Weighted BERT Model

Improvement of Short Text Clustering Based on Weighted Word Embeddings

Weighted K-Means Algorithm Based Text Clustering

The performance of BERT as data representation of text clustering

Research on Text Clustering Based on Concept Weight

Document Representation Methods for Clustering Bilingual Documents

CWC: A Clustering-Based Feature Weighting Approach for Text Classification

A Document Ensemble Clustering Approach Via Dimensionality Reduction

Text Feature Generation Algorithm Based on Clustering Weighted

Text clustering based on term weights automatic partition

A comparative study of feature weighting methods for document co-clustering.

Unsupervised Feature Weighting Based On Local Feature Relatedness

Subspace clustering of text documents with feature weighting k-means algorithm

A Novel Weighted Phrase-Based Similarity for Web Documents Clustering.

Efficient text representation via weighted co-contributions of terms on class tendency

Discovering Thematically Coherent Biomedical Documents Using Contextualized Bidirectional Encoder Representations from Transformers-Based Clustering

A frequent term-based multiple clustering approach for text documents

Semantic Feature Reduction in Chinese Document Clustering

Improvement of Feature Weight Formula in Text Clustering

Document Representation and Clustering Models for Bilingual Documents Clustering.

A Weighted Word Embedding Model for Text Classification