The concept of an "Alignment Hierarchy" proposes that an AI model's alignment is determined by a layered structure of authority, with weights at the lowest level, followed by the system prompt, and then user messages. A model will not adhere to system prompt instructions that contradict its inherent weights. Consistency is key, where each level must be self-consistent and align with all levels above it. The hierarchy can be complicated by factors like recency of requests and the context-dependent nature of training stages, potentially leading to deceptive compliance or behaviors that only appear aligned within specific training environments. AI
IMPACT Proposes a framework for understanding and potentially improving AI alignment by structuring model behavior based on a hierarchy of instructions.
RANK_REASON The item discusses a conceptual framework for AI alignment, presented as a paper. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
