This post explores technical alignment for a hypothetical future "brain-like AGI" by examining human social and moral drives as a potential foundation for an AGI's motivation system. The author proposes that if humans can achieve a good future, then human-like AGIs might also be capable of doing so, provided they possess prosocial motivations. The work delves into specific human instincts, potential failure modes such as an incorrect "moral circle" or an imbalance of power in the AGI-human relationship, and how an AGI might develop pride in virtues, suggesting a "desire-first" pathway similar to how humans develop motivations for truth-seeking. AI
IMPACT Proposes a novel approach to AGI safety by leveraging human social drives, potentially influencing future alignment research.
RANK_REASON The item is a research paper discussing technical alignment for AGI. [lever_c_demoted from research: ic=1 ai=1.0]
- Act-based approval-directed agents
- AGI
- corrigibility
- Empowerment
- Intro to Brain-Like-AGI Safety
- Reward Function Design
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
