DuDi framework boosts small language models' multilingual abilities

Researchers have developed DuDi, a novel dual-signal distillation framework designed to enhance the multilingual capabilities of small language models (SLMs). This method combines sequence-level and token-level signals, incorporating a cross-lingual verbalizer to refine teacher feedback. Experiments demonstrate that DuDi significantly improves performance on Southeast Asian languages, outperforming existing distillation techniques across various model scales and families. AI

影响 Enhances multilingual capabilities of small language models, potentially improving accessibility and performance for under-resourced languages.

排序理由 This is a research paper describing a new method for improving language models. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.CL 阅读 →

• DuDi
• Patomporn Payoungkhamdee
• Southeast Asian languages

• 论文
• 模型发布

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →