本文介绍了用于压缩领域特定应用（重点是量化金融）的大语言模型（LLM）的经验缩放定律。它量化了在迭代剪枝过程中，数据集大小、压缩率和监督格式如何影响专业知识和通用知识领域的性能。研究表明，虽然领域内任务质量会按预期下降，但通用知识基准的性能会更早地崩溃，而思维链监督对于恢复被擦除的通用知识至关重要。 AI
影响 为在资源受限的领域特定应用中部署大语言模型提供了优化框架。
排序理由 该集群包含一篇详细介绍大语言模型蒸馏经验缩放定律的研究论文。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2606.24747v1 Announce Type: new Abstract: Large Language Models (LLMs) achieve strong performance across a growing range of domains, yet their scale poses deployment challenges in applications where latency and cost constraints are critical. This paper derives empirical sca…
Large Language Models (LLMs) achieve strong performance across a growing range of domains, yet their scale poses deployment challenges in applications where latency and cost constraints are critical. This paper derives empirical scaling laws for domain-specific LLM compression, q…