Abstract:Text clustering is an important method for organising the increasing volume of digital content, aiding in the structuring and discovery of hidden patterns in uncategorised data. The effectiveness of text clustering largely depends on the selection of textual embeddings and clustering algorithms. This study argues that recent advancements in large language models (LLMs) have the potential to enhance this task. The research investigates how different textual embeddings, particularly those utilised in LLMs, and various clustering algorithms influence the clustering of text datasets. A series of experiments were conducted to evaluate the impact of embeddings on clustering results, the role of dimensionality reduction through summarisation, and the adjustment of model size. The findings indicate that LLM embeddings are superior at capturing subtleties in structured language. OpenAI's GPT-3.5 Turbo model yields better results in three out of five clustering metrics across most tested datasets. Most LLM embeddings show improvements in cluster purity and provide a more informative silhouette score, reflecting a refined structural understanding of text data compared to traditional methods. Among the more lightweight models, BERT demonstrates leading performance. Additionally, it was observed that increasing model dimensionality and employing summarisation techniques do not consistently enhance clustering efficiency, suggesting that these strategies require careful consideration for practical application. These results highlight a complex balance between the need for refined text representation and computational feasibility in text clustering applications. This study extends traditional text clustering frameworks by integrating embeddings from LLMs, offering improved methodologies and suggesting new avenues for future research in various types of textual analysis.

Text Clustering with Large Language Model Embeddings

ClusterLLM: Large Language Models as a Guide for Text Clustering

Text Clustering as Classification with LLMs

Human-interpretable clustering of short-text using large language models

Large Language Models Enable Few-Shot Clustering

ERASMO: Leveraging Large Language Models for Enhanced Clustering Segmentation

German Text Embedding Clustering Benchmark

Evaluation of Text Cluster Naming with Generative Large Language Models

Large Language Model Enhanced Clustering for News Event Detection

An integrated clustering and BERT framework for improved topic modeling

Graph-Convolutional Networks: Named Entity Recognition and Large Language Model Embedding in Document Clustering

TeC: A Novel Method for Text Clustering with Large Language Models Guidance and Weakly-Supervised Contrastive Learning

Context-Aware Clustering using Large Language Models

Smaller Text Classifiers with Discriminative Cluster Embeddings

A Vector Reconstruction Based Clustering Algorithm Particularly for Large-Scale Text Collection

Short text clustering based on word embeddings and EMD

Enhancing Embedding Performance through Large Language Model-based Text Enrichment and Rewriting

ZeroDL: Zero-shot Distribution Learning for Text Clustering via Large Language Models

Text embedding techniques for efficient clustering of twitter data

Expanding the Text Classification Toolbox with Cross-Lingual Embeddings

Exploring the Upper Limits of Text-Based Collaborative Filtering Using Large Language Models: Discoveries and Insights