PulseAugur
中
实时 22:25:34
English(EN) Chorus: Shared N-gram Embeddings with Layer-wise Adaptation

Chorus方法将N-gram嵌入内存使用量削减87%

研究人员推出了一种新颖的语言模型N-gram嵌入方法Chorus,该方法在保持或提高性能的同时显著降低了内存使用量。与之前为每层使用单独的N-gram嵌入表的方法不同,Chorus采用单一共享的表集,各层通过自己的投影和重缩放机制对其进行自适应。这种共享方法实现了跨层的知识重用,并产生了显著更小的表,其参数量仅为FactorEngram和DeepSeek's Engram等现有方法的一小部分,但性能更优。此外,Chorus与Ember等内存高效优化器兼容,进一步降低了计算开销。 AI

影响 通过降低N-gram嵌入的内存需求,实现了语言模型更高效的扩展。

排序理由 详细介绍语言模型N-gram嵌入新方法的学术论文。

在 Hugging Face Daily Papers 阅读 →

AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →

Chorus方法将N-gram嵌入内存使用量削减87%

本文如何被排名

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
详细介绍语言模型N-gram嵌入新方法的学术论文。
Source corroboration
2 independent sources
Multiple independent publishers reporting the same story raises confidence that it's real and newsworthy.
Topics
paper, infra
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
4 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

完整方法见我们的编辑标准。

报道来源 [2]

  1. arXiv cs.CL TIER_1 English(EN) · Stephen Chung, Wenyu Du ·

    Chorus: 共享N-gram嵌入与层级自适应

    arXiv:2611.23386v2 Announce Type: replace Abstract: Language models can gain capacity at little computational cost from n-gram embedding tables: large tables of learned vectors for short sequences of consecutive tokens, which are looked up and added to the model's input or to its…

  2. Hugging Face Daily Papers TIER_1 English(EN) ·

    Chorus: 具有层级自适应的共享N-gram嵌入

    Chorus shares one set of n-gram embedding tables across all layers, letting each layer project and rescale the embedding using its hidden state, achieving better quality with much smaller tables and lower optimizer memory.