Brief

A critique of Yoshua Bengio's "Scientist AI" proposal raises concerns about its alignment failures and practical feasibility. The author argues that preventing the AI from exploring agentically, a key aspect of scientific discovery, would hinder its progress and potentially lead to unsafe outcomes. Furthermore, the proposed method of training based on associative probabilities, rather than true causal inference, is seen as a fundamental limitation. Despite these criticisms, the author acknowledges the value of Bengio's short-term plan to fine-tune LLMs for identifying potential risks in user requests and appreciates the framing of "anytime preparedness." AI