Researchers have developed a new benchmark called AccessEval to measure ableism bias in large language models (LLMs). This framework addresses subtle biases like response avoidance and cautious framing, which were not captured by previous studies. AccessEval utilizes paired neutral and disability-aware queries across various domains and disability types, evaluating LLM responses using metrics such as VADER score, Regard Score, and an LLM-as-Judge approach. AI
IMPACT This benchmark could lead to more equitable LLM development by identifying and mitigating ableism.
RANK_REASON The cluster describes a new benchmark and research paper focused on evaluating bias in LLMs. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
