PulseAugur
中
实时 18:38:07
English(EN) Rebalancing Token Importance in Language Models with TF-IDF Weighted Cross-Entropy Loss

新方法压缩大语言模型序列以提高效率和语义恢复能力

研究人员开发了新的方法来压缩大语言模型(LLMs)中的序列,以降低计算成本并提高效率。FastE 是一种无需训练的方法,通过识别和移除冗余状态来压缩 Qwen3-Embedding 等嵌入模型中的前缀状态,在性能损失极小的情况下实现了显著的 FLOPs 降低。K-Token Merging 在潜在嵌入空间中操作,合并连续的 Token 嵌入以缩短序列长度并减轻大语言模型的计算负载,在各种基准测试中表现强劲。TokCode 提供了一个用于生成式语义通信中鲁棒语义恢复的框架,通过重构语义域中的冗余并使用轻量级适配器和蒸馏方法来增强擦除弹性。 AI

影响 这些技术可以显著降低处理大语言模型长序列的计算和内存需求,从而实现更高效的部署和更广泛的可访问性。

排序理由 多篇 arXiv 论文介绍了用于大语言模型序列压缩和语义恢复的新颖技术。

在 Hugging Face Daily Papers 阅读 →

AI 生成摘要 · Google Gemini · 来自 5 个来源。 我们如何撰写摘要 →

新方法压缩大语言模型序列以提高效率和语义恢复能力

本文如何被排名

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
多篇 arXiv 论文介绍了用于大语言模型序列压缩和语义恢复的新颖技术。
Source corroboration
5 independent sources
Strong cross-source corroboration — multiple independent publishers covered this within the clustering window.
Topics
paper, infra
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
73 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.
Coverage growth since scoring
+2 source(s) since last score
New sources have picked up this story since our last re-score. Score will update on the next scoring pass.

完整方法见我们的编辑标准。

报道来源 [5]

  1. arXiv cs.CL TIER_1 English(EN) · Zhijian Li, Stefan Larson, Kevin Leach ·

    使用TF-IDF加权交叉熵损失重新平衡语言模型中的Token重要性

    arXiv:2609.11029v1 Announce Type: new Abstract: Large language models are typically trained under uniform token weighting, which allows frequent and low-information tokens to dominate learning and can increase the tendency to memorize surface-level text spans. To address this, we…

  2. arXiv cs.AI TIER_1 English(EN) · Jinsong Shu, Jinyong Wen, Baokun Wang, Zhongle Xie, Lidan Shou, Weiqiang Wang, Gang Chen ·

    FastE:用于 LLM 嵌入推理的读出触发式令牌压缩

    arXiv:2609.08407v1 Announce Type: new Abstract: In this study, we identify depth-dependent prefix redundancy in final-readout LLM embedding models, notably across representative backbones including Qwen3-Embedding and Qwen3-VL-Embedding. We find that removing prefix states is sub…

  3. arXiv cs.AI TIER_1 English(EN) · Zihao Xu, John Harvill, Ziwei Fan, Yizhou Sun, Hao Ding, Hao Wang ·

    在潜在嵌入空间中压缩序列:大型语言模型的 $K$-Token 合并

    arXiv:2604.15153v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) incur significant computational and memory costs when processing long prompts, as full self-attention scales quadratically with input length. Token compression aims to address this challenge by…

  4. arXiv cs.LG TIER_1 English(EN) · Jingzhi Hu, Ouya Wang, Geoffrey Ye Li ·

    用于语义恢复的Token编码

    arXiv:2604.12931v2 Announce Type: replace-cross Abstract: In generative semantic communication, semantic tokens guide receiver-side generative models to synthesize high-dimensional content. In challenging network environments, however, frequent token erasure distorts the conveyed…

  5. Hugging Face Daily Papers TIER_1 English(EN) ·

    使用TF-IDF加权交叉熵损失重新平衡语言模型中的Token重要性

    Large language models are typically trained under uniform token weighting, which allows frequent and low-information tokens to dominate learning and can increase the tendency to memorize surface-level text spans. To address this, we present an information-weighted cross-entropy l…