Richard Ngo
PulseAugur coverage of Richard Ngo — every cluster mentioning Richard Ngo across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
AI Agent Design Explores Social Theory Connections
Richard Ngo's recent post on LessWrong explores the intersection of social theory and agent foundations, drawing connections between human social dynamics and the development of artificial intelligence agents. The piece…
-
AI alignment strategy proposed to prevent rogue AI behavior
Paul Christiano and Richard Ngo propose a method to prevent advanced AI systems from acting against human interests. Their approach involves training AI models to be steerable by a specific, limited set of human-approve…
-
Author argues self-deception claims don't preclude conversation
The author argues against the idea that declaring someone self-deceived inherently prevents conversation. They contend that even if one has privileged access to their own beliefs and motivations, external observers can …
-
AI researchers urged to refuse unethical work over resignation
Employees at AI labs should consider refusing to work on projects they deem morally objectionable, rather than resigning, to exert greater social and financial pressure. This tactic, the author argues, would be more cos…
-
AI Safety Field Urged to Embrace Pragmatism and Power
The author argues that the AI safety field should focus on pragmatism rather than purely intellectual pursuits, especially given the massive investment in AI. They suggest working with existing power structures and capi…
-
Autonomous agents' existential risk debated amid rapid adaptation concerns
A recent discussion on LessWrong explores the potential risks and mitigations associated with Autonomous Replicating Agents (ARAs). While some argue that open-source ARA models or leaks could lead to a point of no retur…
-
AI Safety Researcher Details Career Choices and Motivations
The author reflects on their career path, emphasizing a strong inclination towards technical AI safety research. They detail a progression through various roles, including PhD candidate, fellow at MATS, independent rese…
-
Evolutionary principles mirrored in neural network feature learning
A recent post on LessWrong draws a mathematical analogy between evolutionary processes and neural networks, specifically focusing on inductive biases. The author posits that evolution, like machine learning models, sele…
-
AI researchers debate 'P' with high probability assignments · 1 source tracked
A group of AI researchers and figures, including Daniel Kokotajlo, Ryan Greenblatt, and Joe Carlsmith, are discussing and assigning probabilities to an event or concept referred to as "P." While the exact nature of "P" …
-
AI safety framework expands 'beliefs' beyond probability distributions
Richard Ngo's post introduces a framework for "imprecise beliefs" that extends beyond traditional probability distributions, particularly for AI safety applications. The proposed model, developed by "davidad," defines b…
-
Probabilistic models retain uncertainty even with complete data
This article delves into the concept of inevitable uncertainty in probabilistic world models, using examples like a fish pond and an ideal gas to illustrate the point. The author discusses how even when all data about a…
-
Lawyers urged to join AI safety efforts to mitigate catastrophic risks
A barrister with over a decade of experience in litigation and human rights is advocating for increased involvement of legal professionals in the field of AI safety. The author suggests that lawyers can contribute by he…
-
LessWrong authors defend AI development plan against criticism
LessWrong authors are responding to criticism of their "Plan A" proposal for AI development. They argue that Séb Krier's critique mischaracterizes their plan, which they describe as iterative and transparent, allowing f…
-
AI Thinker Archetypes: Dialectician, Gardener, and Artist
This article presents a romanticized taxonomy of different types of thinkers, drawing inspiration from the MATS 9.1 extension program. It outlines three distinct archetypes: the dialectician, who actively seeks out deba…
-
AI Safety: Future Forecasts Depend on Agent Models, Not Just Compute
This article explores the concept of process alignment in AI safety, arguing that forecasts about the future of AI are heavily influenced by underlying assumptions about the nature of future agents. It posits that our c…
-
AI future predictions flawed by agent assumptions, author argues
The author argues that current AI future predictions are undermined by implicit assumptions about the nature of future agents, whether they be humans, institutions, or AI models. These ontological commitments shape theo…
-
AI Labs Hire Philosophers to Tackle Ethics and Safety
Major AI companies like OpenAI, Anthropic, and Google DeepMind are increasingly employing philosophers to address complex ethical and safety challenges in model development. These experts are tasked with navigating issu…
-
AI researchers discuss superintelligence, alignment, and open-source competition
Several AI researchers and practitioners are sharing their perspectives on various aspects of AI development and safety. Jérémy Perret is revisiting discussions on superintelligence and LLMs, while Arb is detailing supp…
-
AI safety generalists offered curated reading list for growth
A reading list has been compiled for generalists in the AI safety community, aiming to foster growth and project ownership. The list includes 18 essays and blog posts, with frequent citations from Paul Graham, Ben Kuhn,…
-
RLHF updates LLM weights differently than SFT, research finds
New research suggests that Reinforcement Learning from Human Feedback (RLHF) updates LLM weights differently than pre-training or supervised fine-tuning. These RLHF updates are more sparse and tend to rotate the model's…