PulseAugur
中
实时 11:57:39
English(EN) Remembering Distinct Items, Not Tokens: A Learnable Dirichlet-Process Cache Between State-Space Models and Attention

新的狄利克雷过程缓存存储独特信息，性能优于注意力机制

作者 PulseAugur 编辑部 · [4 个来源] ·

研究人员为序列模型开发了一种新颖的记忆系统，该系统存储独特信息而非单个标记，解决了固定状态模型的局限性和注意力机制的计算成本问题。该系统基于可学习的狄利克雷过程缓存，仅为新颖输入分配内存槽，使其能够随着遇到的独特项目数量进行扩展。实验表明，这种方法在显著减少内存使用量的情况下，匹配了全注意力机制的召回性能，尤其是在处理长而冗余的数据流时。 AI

影响 这项研究可能带来更高效的AI模型，通过减少内存开销来处理更长的上下文。

排序理由 该集群包含两篇详细介绍AI模型新颖记忆机制的学术论文。

在 arXiv cs.NE (Neural & Evolutionary) 阅读 →

AI 生成摘要 · Google Gemini · 来自 4 个来源。 我们如何撰写摘要 →

新的狄利克雷过程缓存存储独特信息，性能优于注意力机制

报道来源 [4]

  1. arXiv cs.AI TIER_1 English(EN) · Siddharth Pal, Viktoria Rojkova ·

    Remembering Distinct Items, Not Tokens: A Learnable Dirichlet-Process Cache Between State-Space Models and Attention

    arXiv:2607.09889v1 Announce Type: cross Abstract: Fixed-state sequence models compress an unbounded past into a bounded state, which caps their associative recall at roughly the state dimension; attention escapes the cap by keeping a key-value entry for every token, at quadratic …

  2. arXiv cs.AI TIER_1 English(EN) · Siddharth Pal, Viktoria Rojkova ·

    Context by Distinct Information: An Auditable Dirichlet-Process Working Memory for Long, Redundant Context Streams

    arXiv:2607.10441v1 Announce Type: cross Abstract: Context engineering decides what information a model carries forward, and current designs meter it in tokens: compressing the past into a bounded recurrent state, keeping a key-value entry for every token, or imposing a fixed budg…

  3. arXiv cs.IR (Information Retrieval) TIER_1 English(EN) · Viktoria Rojkova ·

    Context by Distinct Information: An Auditable Dirichlet-Process Working Memory for Long, Redundant Context Streams

    Context engineering decides what information a model carries forward, and current designs meter it in tokens: compressing the past into a bounded recurrent state, keeping a key-value entry for every token, or imposing a fixed budget through a window or eviction rule. All three ma…

  4. arXiv cs.NE (Neural & Evolutionary) TIER_1 English(EN) · Viktoria Rojkova ·

    Remembering Distinct Items, Not Tokens: A Learnable Dirichlet-Process Cache Between State-Space Models and Attention

    Fixed-state sequence models compress an unbounded past into a bounded state, which caps their associative recall at roughly the state dimension; attention escapes the cap by keeping a key-value entry for every token, at quadratic compute and a cache that grows with the sequence. …

相关实体

相关话题