English(EN) Stochastic Rounding in Low-Precision Transformer Inference: A Variable-Precision Emulation Study of a Small GPT-2

随机舍入可改善低精度Transformer推理

作者 PulseAugur 编辑部 · [1 个来源] · 2026-10-03 04:00

研究人员调查了在低精度Transformer推理中使用随机舍入（SR）与就近舍入（RN）的对比，发现最佳方法取决于模型中的特定层。他们开发了一种可变精度随机舍入（VPSR）算法，以便在任意精度下进行实验。他们的分析显示，SR的误差包络比RN增长得慢，尤其是在多层感知机（MLP）中，而RN更适合语言模型头。通过将SR策略性地应用于MLP，将RN应用于头部，他们在DistilGPT-2上实现了接近全精度的困惑度。 AI

影响优化低精度推理可以实现资源受限硬件上大型语言模型的更高效部署。

排序理由学术论文，详细介绍了优化Transformer推理的新算法和实验结果。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 4 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 学术论文，详细介绍了优化Transformer推理的新算法和实验结果。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, infra; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Same-day; Cluster formed today. Ranking reflects the current source set at time of score.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.LG TIER_1 English(EN) · Yohan Chatelain (Krembil Centre for Neuroinformatics, CAMH, Toronto, Canada), Pablo de Oliveira Castro (Universite Paris-Saclay, UVSQ, LI-PaRAD, Versailles, France) · 2026-10-03 04:00

低精度Transformer推理中的随机舍入：小型GPT-2的可变精度模拟研究

arXiv:2610.01889v1 Announce Type: new Abstract: Should low-precision transformer inference use stochastic rounding (SR) or round-to-nearest (RN)? The answer depends on where in the network you look. We isolate this effect by holding the numerical format fixed and varying only the…

本文如何被排名

报道来源 [1]

低精度Transformer推理中的随机舍入：小型GPT-2的可变精度模拟研究

相关实体

相关话题