PulseAugur
中
实时 05:39:14
English(EN) I trained a 75M parameter LLM from scratch on 18B tokens and it beats a model almost double its size

开发者训练出拥有7500万参数的大型语言模型,性能超越更大模型

一位开发者从头开始训练了一个名为KeyLM的7500万参数语言模型,并使用了180亿个token进行预训练。KeyLM的指令调优版本在IFEval基准测试中表现优于SmolLM-135M-Instruct,尽管其参数量和训练数据量都显著少于后者。虽然KeyLM在指令遵循方面表现出色,但在其他基准测试中,其表现与其规模相符,并且在知识类任务中存在频繁幻觉的问题。 AI

影响 展示了针对特定任务高效训练小型模型的可能性,可能降低定制化大型语言模型开发的门槛。

排序理由 一位独立开发者发布了一个自定义训练的模型及其基准测试结果。[lever_c_demoted from research: ic=1 ai=1.0]

在 r/LocalLLaMA 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →

开发者训练出拥有7500万参数的大型语言模型,性能超越更大模型

本文如何被排名

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
一位独立开发者发布了一个自定义训练的模型及其基准测试结果。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration
Single-source cluster
Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics
model release, paper
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
170 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

完整方法见我们的编辑标准。

报道来源 [1]

  1. r/LocalLLaMA TIER_1 English(EN) · /u/cakes_and_candles ·

    我从头开始用180亿个token训练了一个7500万参数的大语言模型,它的表现优于一个规模近乎两倍的模型

    <!-- SC_OFF --><div class="md"><p>I trained a small language model from scratch called KeyLM. It is 75M params, decoder-only, and there is a pretrained base, an instruction-tuned version, and a GGUF.</p> <p>On IFEval (instruction following) the 75M instruct model scores slightly …