Johnson-Chvatal heuristic

New clustering method slashes LLM inference costs by 50x

Researchers have developed a novel two-stage clustering algorithm designed to significantly reduce the inference costs and latency associated with large language models (LLMs). This method, which uses Mini-batch K-Means…