LLMs can achieve distributional alignment but generalization is unpredictable

Researchers have investigated how to achieve distributional alignment in large language models (LLMs) for applications like simulating human behavior. Their study found that LLMs can align their output distributions with reference human distributions and generalize to unseen questions within the same domain. However, generalization across different topics is dependent on the relationship between those topics, and alignment can unexpectedly degrade the LLM's capabilities, such as its creativity, indicating unpredictable and unstable generalization properties. AI

IMPACT This research highlights potential limitations in LLM generalization and the unpredictable nature of alignment, suggesting caution for applications relying on simulated human behavior.

RANK_REASON The item is an academic paper detailing research findings on LLM capabilities. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• alphaXiv
• arXiv
• CatalyzeX Code Finder for Papers
• Connected Papers
• CORE Recommender
• DagsHub
• Gotit.pub
• Hugging Face
• Influence Flower
• Large language models
• Litmaps
• ScienceCast
• scite Smart Citations

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →