Diffusion language models show mixed results in factual grounding for text generation

A new research paper explores the faithfulness of diffusion language models in table-to-text generation, comparing them against autoregressive models like GPT-2 and GPT-Neo. The study found that while diffusion models like LLaDA can achieve higher text similarity scores (BLEU), they do not necessarily improve factual grounding against the source table. Fine-tuning LLaDA with QLoRA further boosted BLEU scores but did not enhance faithfulness, indicating that reference-based metrics alone are insufficient for evaluating this aspect of generation. AI

影响 Highlights the limitations of current metrics for evaluating factual grounding in generative models, suggesting a need for more robust evaluation methods.

排序理由 Research paper published on arXiv detailing a diagnostic study of language models. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.CL 阅读 →

• arXiv
• Diffusion language models
• GPT-2
• GPT-Neo
• Hugging Face
• Noussayba El Marrakchi
• QLoRA
• ToTTo

• 论文
• 模型发布

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →