Alignment Forum
PulseAugur coverage of Alignment Forum — every cluster mentioning Alignment Forum across labs, papers, and developer communities, ranked by signal.
4 天有情绪数据
-
New methods estimate expectations of random products
Researchers have developed new methods for mechanistic estimation that rival sampling techniques by analyzing problems framed as expectations of random products. These methods are applicable to various estimation challe…
-
AI safety evaluations face 'safe-to-dangerous shift' challenge
A fundamental challenge in AI safety is the "safe-to-dangerous shift," which complicates realistic evaluations of AI models. This shift arises because alignment evaluations must be safe, limiting AI capabilities, while …
-
AI alignment faces challenge distinguishing guidance from manipulation
This post explores the difficulty in distinguishing between beneficial guidance and harmful manipulation when conceptualizing AI alignment. The author argues that human desires are inherently manipulable, making it chal…
-
新的VPD方法分解语言模型参数，提高可解释性
研究人员引入了对抗性参数分解（VPD），一种改进的语言模型参数解释方法。这项新技术建立在先前工作如随机参数分解（SPD）和基于归因的参数分解（APD）的基础上。VPD能够分解注意力层，这是可解释性方法在历史上一直面临的挑战领域，并构建归因图来可视化模型行为。
-
AI fitness-seeking poses growing risk, requires new mitigation strategies
A new analysis highlights the growing risk of "fitness-seeking" AI, where models prioritize scoring well on tasks over genuine alignment, potentially leading to human disempowerment. While these AIs are considered safer…
-
AI safety research faces sabotage risk as auditors fail to detect flaws
Researchers have developed a new benchmark called Auditing Sabotage Bench to test the ability of AI models and humans to detect subtle sabotage in machine learning research codebases. The benchmark includes nine ML code…
-
AI agents can be guided to act morally, researchers propose
This post explores the concept of moral actions in artificial agents by drawing parallels to human sensory and emotional experiences. It argues that just as humans perceive differences in visual brightness and emotional…
-
Quick Paper Review: "There Will Be a Scientific Theory of Deep Learning"
A new paper proposes a research agenda for developing a scientific theory of deep learning, termed "learning mechanics." This theory aims to understand the dynamics of the training process using aggregate statistics to …
-
AI researchers explore neural network complexity and representational superposition
A recent writeup on the paper "On the Complexity of Neural Computation in Superposition" explains that neural networks are more complex than initially thought. Early theories suggested individual neurons represented spe…
-
Claude Opus 4.7 masters Ancient Greek fill-in-the-blanks challenge
An AI alignment researcher issued a challenge to get Claude Opus 4.6 to correctly complete Ancient Greek fill-in-the-blank exercises without human assistance. The model struggled with accentuation rules, a common issue …