English(EN) I trained a 102M recursive BitNet-v2 model from scratch: 64K context, trained on less than 5B tokens

新发布的102M BitNet-v2模型支持64K上下文训练

作者 PulseAugur 编辑部 · [1 个来源] · 2026-10-11 10:19

一款名为Recursive BitNet N-Gram 102M的新型102M参数模型已发布，该模型结合了三元权重、共享Transformer层和哈希n-gram嵌入。这个实验性模型是从头开始在约47亿token上训练的，其训练的很大一部分集中在64K上下文窗口上。虽然基准测试结果显示效果一般，但该模型在多项评估中的平均得分为40.59%，开发者也欢迎反馈。 AI

影响这个实验性模型展示了用于高效训练和长上下文窗口的新颖技术，可能影响未来小型模型的发展。

排序理由发布了一个新的、尽管是实验性的模型，具有新颖的架构特性和基准测试结果。[lever_c_demoted from research: ic=1 ai=1.0]

在 r/LocalLLaMA 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 1 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 发布了一个新的、尽管是实验性的模型，具有新颖的架构特性和基准测试结果。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: model release, other; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: Same-day; Cluster formed today. Ranking reflects the current source set at time of score.

完整方法见我们的编辑标准。

报道来源 [1]

r/LocalLLaMA TIER_1 English(EN) · /u/Illustrious-Fig-2280 · 2026-10-11 10:19

我从头训练了一个102M递归BitNet-v2模型：64K上下文，在不到5B token上训练

<div class="md">DISCLAIMER: the post and the model card was made with the assist of AI. Hiya, I’m releasing Recursive BitNet N-Gram 102M, a small experiment combining ternary weights, shared transformer layers, and hashed n-gram embeddings. Weight…

本文如何被排名

报道来源 [1]

我从头训练了一个102M递归BitNet-v2模型：64K上下文，在不到5B token上训练

相关实体

相关话题