Zico Kolter
PulseAugur coverage of Zico Kolter — every cluster mentioning Zico Kolter across labs, papers, and developer communities, ranked by signal.
-
AI refusal mechanisms are unreliable and could be misused, experts warn
Current large language models are trained to refuse dangerous or harmful requests, a capability that has become a core tenet of AI safety. However, this refusal mechanism is not foolproof and can fail, potentially leadi…
-
AI safety advocate Paul Christiano joins OpenAI board amid safety concerns
Paul Christiano, a prominent AI safety researcher, is joining OpenAI's Foundation board, focusing on mitigating existential risks from advanced AI. His appointment comes amid increased scrutiny of OpenAI's safety protoc…
-
Autonomous agents risk double-purchasing due to system design flaws · 2 sources tracked
A critical vulnerability has been identified in autonomous agent commerce systems, where agents can inadvertently make duplicate purchases. This issue arises from a combination of factors: agents' inherent retry logic f…
-
Gray Swan tackles AI security risks with advanced red teaming tools
Gray Swan, a company co-founded by Zico Kolter and Matt Fredrikson, is developing tools and methodologies for AI security, focusing on "gray swan" events—unlikely but foreseeable AI incidents. Their work addresses vulne…
-
AI accelerates cyberattacks, worsening workforce shortage
The cybersecurity workforce faces a critical shortage, exacerbated by AI-powered attacks that are becoming faster and more autonomous. While interest in cybersecurity is high among teen girls, many do not pursue careers…
-
AI security firm Gray Swan raises $40M at $200M valuation
AI startup Gray Swan has secured $40 million in Series A funding, valuing the company at $200 million. The startup employs approximately 15,000 security professionals who act as hackers to pressure-test AI models from m…
-
OpenAI forms independent board committee for AI safety and security oversight
OpenAI has established an independent Safety and Security Committee, chaired by Zico Kolter, to oversee critical safety and security measures for its model development and deployment. This committee will review safety e…
-
OpenAI adds AI safety expert Zico Kolter to its Board of Directors
OpenAI has appointed Zico Kolter, a professor and head of Carnegie Mellon University's Machine Learning Department, to its Board of Directors. Kolter's expertise in AI safety, alignment, and model robustness is expected…