Abstract:Background: Internet provides different tools for communicating with patients, such as social media (e.g., Twitter) and email platforms. These platforms provided new data sources to shed lights on patient experiences with health care and improve our understanding of patient-provider communication. Several existing topic modeling and document clustering methods have been adapted to analyze these new free-text data automatically. However, both tweets and emails are often composed of short texts; and existing topic modeling and clustering approaches have suboptimal performance on these short texts. Moreover, research over health-related short texts using these methods has become difficult to reproduce and benchmark, partially due to the absence of a detailed comparison of state-of-the-art topic modeling and clustering methods on these short texts. Methods: We trained eight state -of-the-art topic modeling and clustering algorithms on short texts from two health-related datasets (tweets and emails): Latent Semantic Indexing (LSI), Latent Dirichlet Allocation (LDA), LDA with Gibbs Sampling (GibbsLDA), Online LDA, Biterm Model (BTM), Online Twitter LDA, and Gibbs Sampling for Dirichlet Multinomial Mixture (GSDMM), as well as the k-means clustering algorithm with two different feature representations: TF-IDF and Doc2Vec. We used cluster validity indices to evaluate the performance of topic modeling and clustering: two internal indices (i.e. assessing the goodness of a clustering structure without external information) and five external indices (i.e. comparing the results of a cluster analysis to an externally known provided class labels). Results: In overall, for number of clusters (k) from 2 to 50, Online Twitter LDA and GSDMM achieved the best performance in terms of internal indices, while LSI and k-means with TF-IDF had the highest external indices. Also, of all tweets (N = 286, 971; HPV represents 94.6% of tweets and lynch syndrome represents 5.4%), for k = 2, most of the methods could respect this initial clustering distribution. However, we found model performance varies with the source of data and hyper-parameters such as the number of topics and the number of iterations used to train the models. We also conducted an error analysis using the Hamming loss metric, for which the poorest value was obtained by GSDMM on both datasets. Conclusions: Researchers hoping to group or classify health related short-text data can expect to select the most suitable topic modeling and clustering methods for their specific research questions. Therefore, we presented a comparison of the most common used topic modeling and clustering algorithms over two health-related, short text datasets using both internal and external clustering validation indices. Internal indices suggested Online Twitter LDA and GSDMM as the best, while external indices suggested LSI and k-means with TF-IDF as the best. In summary, our work suggested researchers can improve their analysis of model performance by using a variety of metrics, since there is not a single best metric.

Visual topic models for healthcare data clustering

Visual Exploration of Internet News Via Sentiment Score and Topic Models

The detection of community health surveillance using distributed semantic assisted non-negative matrix factorization on topic modeling through sentiment analysis

SocialVisTUM: An Interactive Visualization Toolkit for Correlated Neural Topic Models on Social Media Opinion Mining

Evaluation of clustering and topic modeling methods over health-related tweets and emails

Novel Approach on sustainable Topic Modelling paradigms for Recognition of health-related topics in a social platform through Distributed file system

Visualizing Mental Health Insights: A Pipeline from Social Media to Chernoff Faces

Visualizing Nursing Narratives: An Evaluation of Latent Dirichlet Allocation Topic Modeling for Care Reports

Humane Visual AI: Telling the Stories Behind a Medical Condition

Bringing Modern Machine Learning into Clinical Practice Through the Use of Intuitive Visualization and Human–Computer Interaction

Probabilistic neural network based visual data mining for the healthcare sector

ViDi: Descriptive Visual Data Clustering as Radiologist Assistant in COVID-19 Streamline Diagnostic

VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledge

Discovering Thematically Coherent Biomedical Documents Using Contextualized Bidirectional Encoder Representations from Transformers-Based Clustering

Clustering and Topic Modeling over Tweets: A Comparison over a Health Dataset

From Text to Topics in Healthcare Records: An Unsupervised Graph Partitioning Methodology

Modeling Mobile Visualization for Medical Reports of Complex Chronic Diseases

VISTopic: A visual analytics system for making sense of large document collections using hierarchical topic modeling

Network-based modeling and intelligent data mining of social media for improving care

Comparison of Methods for Estimating Temporal Topic Models From Primary Care Clinical Text Data: Retrospective Closed Cohort Study

Enhanced V-Net approach for the emotion recognition and sentiment analysis in the healthcare data