MesaNet 使用最优测试时训练以改进序列建模

作者 PulseAugur 编辑部 · [1 个来源] · 2026-06-04 04:00

研究人员开发了 MesaNet，这是一种新颖的序列建模架构，通过局部最优测试时训练来优化性能。这种方法源于由共轭梯度求解器最小化的上下文内损失，允许分块并行化和可扩展性。实验表明，MesaNet 在序列建模任务上实现了比以往的循环神经网络更低的困惑度（perplexity）和更好的下游性能，尤其是在需要长上下文的任务上，尽管它会增加推理计算量。 AI

影响通过增加测试时计算量，提出了一种改进长上下文任务性能的序列建模新方法。

排序理由该集群包含一篇详细介绍新模型架构及其实验评估的研究论文。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.AI 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 0 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 该集群包含一篇详细介绍新模型架构及其实验评估的研究论文。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, model release; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: 155 days old; Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.AI TIER_1 English(EN) · Johannes von Oswald, Nino Scherrer, Seijin Kobayashi, Luca Versari, Songlin Yang, Sarthak Mittal, Maximilian Schlegel, Kaitlin Maile, Yanick Schimpf, Oliver Sieberling, Alexander Meulemans, Rif A. Saurous, Guillaume Lajoie, Charlotte Frenkel, Razvan Pasc… · 2026-06-04 04:00

MesaNet：通过局部最优测试时训练进行序列建模

arXiv:2506.05233v2 Announce Type: replace-cross Abstract: Sequence modeling is currently dominated by causal transformer architectures that use softmax self-attention. Although widely adopted, transformers require scaling memory and compute linearly during inference. A recent str…

本文如何被排名

报道来源 [1]

MesaNet：通过局部最优测试时训练进行序列建模

相关话题