研究人员开发了潜在上下文语言模型（LCLMs），这是一类新型的编码器-解码器压缩器，旨在解决长上下文语言模型推理中的内存瓶颈问题。通过广泛的架构搜索和在超过3500亿个token上的预训练，这些模型实现了1:4、1:8和1:16的压缩比。LCLMs通过提高通用任务性能、压缩速度和减少峰值内存使用量，优于现有方法，使其成为长视野代理的高效骨干。 AI
影响 引入了一种高效长上下文处理的新方法，有望实现更强大、内存占用更少的AI代理。
排序理由 这是一篇详细介绍新模型架构及其性能的研究论文。
在 Hugging Face Daily Papers 阅读 →
AI 生成摘要 · Google Gemini · 来自 5 个来源。 我们如何撰写摘要 →
arXiv:2606.09659v1 Announce Type: cross Abstract: Long-context language model inference is bottlenecked by memory, as the KV cache grows with context length. Recent techniques to compress the KV cache fall short: they either degrade model quality substantially or require consider…
Long-context language model inference is bottlenecked by memory, as the KV cache grows with context length. Recent techniques to compress the KV cache fall short: they either degrade model quality substantially or require considerable time and compute to compress a single long pr…
Long-context language model inference is bottlenecked by memory, as the KV cache grows with context length. Recent techniques to compress the KV cache fall short: they either degrade model quality substantially or require considerable time and compute to compress a single long pr…
Encoder-decoder compression techniques are improved through architectural search and large-scale pretraining to create Latent Context Language Models that efficiently handle long contexts with better performance and memory usage compared to traditional KV cache methods.
  submitted by   <a href="https://www.reddit.com/user/DeltaSqueezer"> /u/DeltaSqueezer </a> <br /> <span><a href="https://venturebeat.com/data/context-compression-finally-works-in-production-new-research-cuts-llm-input-16x-without-the-accuracy-hit">[link]</a></span>  …