TopicMerge combines LLMs and efficient clustering for scalable topic modeling

Researchers have introduced TopicMerge, a novel two-stage pipeline designed to improve topic modeling by combining efficient clustering with the power of large language models (LLMs). The method first segments documents into numerous small clusters, which are then semantically labeled. Subsequently, an LLM iteratively merges these micro-clusters based on local context, achieving state-of-the-art performance while significantly reducing computational costs. Experiments show TopicMerge can decrease LLM calls by over 92% compared to traditional document-level methods, offering a more scalable and efficient solution for topic analysis. AI

IMPACT Offers a more computationally efficient approach to topic modeling using LLMs, potentially enabling wider adoption in large-scale text analysis.

RANK_REASON The cluster contains a research paper detailing a new method for topic modeling. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• alphaXiv
• arXiv
• Bills dataset
• CatalyzeX
• DagsHub
• Gotit.pub
• Hugging Face
• large language model
• ScienceCast
• TopicMerge
• Wiki dataset

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →