PulseAugur
EN
LIVE 22:13:44

Chorus method slashes n-gram embedding memory use by 87%

Researchers have introduced Chorus, a novel method for n-gram embeddings in language models that significantly reduces memory usage while maintaining or improving performance. Unlike previous approaches that used separate n-gram embedding tables for each layer, Chorus employs a single shared set of tables that each layer adapts through its own projection and rescaling mechanism. This shared approach allows for knowledge reuse across layers and results in substantially smaller tables, outperforming existing methods like FactorEngram and DeepSeek's Engram with a fraction of the parameters. Additionally, Chorus is compatible with memory-efficient optimizers like Ember, further reducing computational overhead. AI

IMPACT Enables more efficient scaling of language models by reducing memory requirements for n-gram embeddings.

RANK_REASON Research paper detailing a new method for n-gram embeddings in language models.

Read on Hugging Face Daily Papers →

AI-generated summary · Google Gemini · from 2 sources. How we write summaries →

Chorus method slashes n-gram embedding memory use by 87%

How we ranked this

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
Research paper detailing a new method for n-gram embeddings in language models.
Source corroboration
2 independent sources
Multiple independent publishers reporting the same story raises confidence that it's real and newsworthy.
Topics
paper, infra
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
4 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

Full methodology in our editorial standards.

COVERAGE [2]

  1. arXiv cs.CL TIER_1 English(EN) · Stephen Chung, Wenyu Du ·

    Chorus: Shared N-gram Embeddings with Layer-wise Adaptation

    arXiv:2611.23386v2 Announce Type: replace Abstract: Language models can gain capacity at little computational cost from n-gram embedding tables: large tables of learned vectors for short sequences of consecutive tokens, which are looked up and added to the model's input or to its…

  2. Hugging Face Daily Papers TIER_1 English(EN) ·

    Chorus: Shared N-gram Embeddings with Layer-wise Adaptation

    Chorus shares one set of n-gram embedding tables across all layers, letting each layer project and rescale the embedding using its hidden state, achieving better quality with much smaller tables and lower optimizer memory.