New method reads and steers internal priorities in language models

Researchers have developed a new method called Constitutional Value Potentials (CVP) to read and steer the internal priorities of language models. CVP learns a scalar potential for each value from a model's hidden state, indicating its internal pressure to preserve that value. This allows for the identification of priority margins, which are crucial for understanding how models handle value conflicts. The system predicts conflict violations with high accuracy and can generalize across different model scales, suggesting that these priorities are accessible within the model's activation space rather than solely through output behavior. AI

IMPACT Enables deeper understanding and control over LLM value alignment, potentially improving safety and reliability.

RANK_REASON The cluster contains a research paper detailing a new method for analyzing language model behavior. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• alphaXiv
• arXiv
• arXivLabs
• CatalyzeX Code Finder for Papers
• Constitutional Value Potentials
• CORE Recommender
• DagsHub
• Gotit.pub
• Hugging Face
• IArxiv Recommender
• Influence Flower
• Qwen2.5
• ScienceCast

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →