MVR-cache

MVR-cache boosts LLM semantic caching hit rates by 37%

Researchers have developed MVR-cache, a new semantic caching system designed to reduce the costs and latency associated with Large Language Models (LLMs). This system utilizes Multi-Vector Retrieval (MVR) and a learnabl…