AI risk aversion generalizes across vast stakes, but not yet reliably

Researchers have developed a new benchmark, RiskAverseOOD, to test how well language models generalize risk aversion from low-stakes scenarios to high-stakes situations. Experiments using various methods on models like Qwen3, Gemma-3, and Llama-3 demonstrated that risk aversion learned at low stakes can partially generalize across vast differences in stakes. While current models show improved risk-averse behavior, they are not yet consistently reliable enough to serve as a failsafe against potential AI misalignment. AI

IMPACT Investigates potential safety mechanisms for AI by testing generalization of risk aversion, crucial for mitigating risks of misalignment.

RANK_REASON Academic paper introducing a new benchmark and experimental results. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• Direct Preference Optimization
• Gemma-3
• Gemma-3-12B-IT
• Llama-3
• Llama-3.1-8B-Instruct
• Qwen3
• Qwen3 14B
• Qwen3 1.7B
• Qwen3_8B
• RiskAverseOOD
• supervised fine-tuning

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →