Researchers quantify and mitigate socially desirable responding in LLMs

Researchers have developed a new framework to identify and reduce socially desirable responding (SDR) in large language models (LLMs) when they are evaluated using self-report questionnaires. This SDR, where models provide preferred answers rather than honest ones, can skew assessment results for persona consistency, safety, and bias. The proposed method quantifies SDR by comparing responses under honest versus fake-good instructions and uses a graded forced-choice inventory to mitigate it, showing significant reduction in SDR while preserving persona recovery. AI

IMPACT Introduces a method to improve the reliability of LLM evaluations, particularly for safety and bias assessments.

RANK_REASON Academic paper introducing a new framework for evaluating LLMs.

Read on arXiv cs.CL →

• Big Five
• Kensuke Okada
• LLMs
• NLP
• IRT

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →