Diffusion Language Models Show Sampling-Dependent Bias

A new research paper explores the fairness of Diffusion Language Models (DLMs), which are emerging as an alternative to autoregressive language models. The study, conducted across LLaDA, LLaDA-1.5, and Dream models using the Open-BBQ benchmark, found that DLMs exhibit social bias. Crucially, the magnitude of this bias and the relative performance of models are highly sensitive to sampling configurations, such as block length and remasking strategy. While standard debiasing techniques can reduce bias without performance degradation, this sampling dependence persists even after mitigation, indicating that DLM fairness must be evaluated in conjunction with their sampling settings. AI

IMPACT Highlights the need for new evaluation methods for diffusion language models to ensure fairness across different sampling configurations.

RANK_REASON Research paper published on arXiv detailing findings about diffusion language models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• autoregressive language models
• Diffusion language models
• dream
• Koshiro Saito
• LLaDA 1.5
• Open-BBQ

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →