研究人员开发了FreshCache，一个新颖的三层语义缓存系统，旨在降低检索增强型大型语言模型（LLM）的延迟和成本。FreshCache将缓存重用视为一个风险约束的时间推理问题，使用拟合的指数衰减模型和学习到的MLP来估计缓存数据过时的概率。与二元的过时/新鲜决策不同，这种方法允许数据随着时间的推移优雅地降级。在基准测试中，FreshCache在最小化过时错误的同时实现了显著的搜索API节省，其性能优于SemanticTTL和vCache等现有方法。 AI
影响 优化LLM检索效率，可能降低AI应用的成本和延迟。
排序理由 该集群描述了一篇关于LLM中语义缓存新颖系统的研究论文。
在 Hugging Face Daily Papers 阅读 →
AI 生成摘要 · Google Gemini · 来自 4 个来源。 我们如何撰写摘要 →
arXiv:2607.04281v1 Announce Type: cross Abstract: Semantic caching reduces the latency and cost of retrieval-augmented generation (RAG) by serving cached answers to semantically similar queries, but most existing methods do not model the time-varying freshness of open-web evidenc…
Semantic caching reduces the latency and cost of retrieval-augmented generation (RAG) by serving cached answers to semantically similar queries, but most existing methods do not model the time-varying freshness of open-web evidence. We present FreshCache, a three-tier semantic ca…
Research addresses cache management for LLM agents by formalizing semantic cache replacement as an online problem with switching costs, proposing SOLAR—a learning-augmented framework that outperforms traditional heuristics through regret-based modification timing and Bayesian con…
<p><a class="article-body-image-wrapper" href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fquzbk59l8l91dcmxj65n.png"><img alt="6 Semantic Cachi…