Abstract:Text clustering is an important method for organising the increasing volume of digital content, aiding in the structuring and discovery of hidden patterns in uncategorised data. The effectiveness of text clustering largely depends on the selection of textual embeddings and clustering algorithms. This study argues that recent advancements in large language models (LLMs) have the potential to enhance this task. The research investigates how different textual embeddings, particularly those utilised in LLMs, and various clustering algorithms influence the clustering of text datasets. A series of experiments were conducted to evaluate the impact of embeddings on clustering results, the role of dimensionality reduction through summarisation, and the adjustment of model size. The findings indicate that LLM embeddings are superior at capturing subtleties in structured language. OpenAI's GPT-3.5 Turbo model yields better results in three out of five clustering metrics across most tested datasets. Most LLM embeddings show improvements in cluster purity and provide a more informative silhouette score, reflecting a refined structural understanding of text data compared to traditional methods. Among the more lightweight models, BERT demonstrates leading performance. Additionally, it was observed that increasing model dimensionality and employing summarisation techniques do not consistently enhance clustering efficiency, suggesting that these strategies require careful consideration for practical application. These results highlight a complex balance between the need for refined text representation and computational feasibility in text clustering applications. This study extends traditional text clustering frameworks by integrating embeddings from LLMs, offering improved methodologies and suggesting new avenues for future research in various types of textual analysis.

ZeroDL: Zero-shot Distribution Learning for Text Clustering via Large Language Models

Zero-shot Model-based Reinforcement Learning using Large Language Models

Zero-Shot Learning Over Large Output Spaces : Utilizing Indirect Knowledge Extraction from Large Language Models

ClusterLLM: Large Language Models as a Guide for Text Clustering

Large Language Models Enable Few-Shot Clustering

Text Clustering as Classification with LLMs

Large Language Models Are Zero-Shot Text Classifiers

Improved Zero-Shot Classification by Adapting VLMs with Text Descriptions

Human-interpretable clustering of short-text using large language models

Enabling Small Models for Zero-Shot Classification through Model Label Learning

Translate to Disambiguate: Zero-shot Multilingual Word Sense Disambiguation with Pretrained Language Models

Open-source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Ranking

Context-Aware Clustering using Large Language Models

Zero-Shot Topic Classification of Column Headers: Leveraging LLMs for Metadata Enrichment

Beyond Prompting: Making Pre-trained Language Models Better Zero-shot Learners by Clustering Representations

Adaptable and Reliable Text Classification using Large Language Models

Clustering in Zero-Resource

Text Clustering with Large Language Model Embeddings

zrLLM: Zero-Shot Relational Learning on Temporal Knowledge Graphs with Large Language Models

A Setwise Approach for Effective and Highly Efficient Zero-shot Ranking with Large Language Models

Pushing Boundaries: Exploring Zero Shot Object Classification with Large Multimodal Models