LLM co-evolution boosted by vocabulary dropout for sustained diversity

Researchers have developed a technique called vocabulary dropout to address diversity collapse in co-evolutionary language model training. This method involves applying a random mask to the proposer model's output logits, preventing it from generating repetitive problems. Experiments with Qwen3-4B and Qwen3-8B models on mathematical reasoning tasks showed that vocabulary dropout maintained proposer diversity and led to significant solver improvements, particularly on challenging benchmarks. AI

影响 Introduces a method to improve LLM training diversity and performance on reasoning tasks.

排序理由 This is a research paper detailing a new technique for LLM training.

在 arXiv cs.CL 阅读 →

• arXiv
• Jacob Dineen
• Qwen3-4B
• Qwen3-8B

• 论文
• 模型发布

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →