AI safety
AI safety coverage moves through three modalities: alignment research papers, incident reports from deployed systems, and policy responses to both. RdyGo's PulseAugur tracks all three — alignment-team blog posts from frontier labs like Anthropic and OpenAI, jailbreak reports, red-teaming results, incident postmortems, and the regulatory responses that shape what labs ship next. The signal we boost: incidents corroborated by multiple independent sources, evaluations from independent groups like Apollo Research and the Alignment Research Center, and policy actions from bodies with enforcement authority — the EU AI Act, the AI Safety Institute, and NIST. The signal we demote: vague concerns, speculation about hypothetical risks, and uncorroborated incident reports.
- Coverage
- 50stories
- Window
- today
- Mix
- tool 38 commentary 7 research 3 significant 2
-
AI agents face production deployment hurdles due to autonomous risks
Developers and enterprises are hesitant to deploy AI agents into production due to concerns about autonomous tool execution, hallucinated logic, and unpredictable behavior. These fears stem from the gap between impressi…
-
Mayo Clinic Accused of Cutting Corners in AI Research
A former executive at Mayo Clinic has accused the institution of cutting corners in its artificial intelligence research. The ex-director of research operations claims she was sidelined and subsequently terminated for v…
-
China warns of Claude AI backdoor; Anthropic refutes claims
China has raised concerns about potential security risks associated with Anthropic's Claude AI model, specifically warning of a "backdoor" in its code. Anthropic has responded to these allegations, refuting the claims a…
-
UK NHS adopts AI blood test for uterine cancer screening
The UK's National Health Service (NHS) is implementing an AI-powered blood test called PinPoint to screen for uterine cancer. This new method, costing approximately £30, has demonstrated a 99% accuracy rate in ruling ou…
-
AI Gateways Enforce LLM Security Guardrails, Bifrost Offers Centralized Control
An AI gateway is crucial for enforcing security guardrails in enterprise LLM applications, protecting against threats like prompt injection and data exfiltration. Bifrost, an open-source gateway developed by Maxim AI, o…
-
Japan's AI data law risks sharing sensitive health info without consent
A proposed amendment to Japan's Act on the Protection of Personal Information is raising concerns about the potential misuse of sensitive health data. The amendment could allow personal health information, such as medic…
-
AI Security & Governance: Agents, Data Sovereignty, and Trust
A series of articles discusses the increasing security and governance challenges associated with AI, particularly concerning AI agents and data sovereignty. Key themes include the prevalence of AI agent security inciden…
-
Prompt Injection and Shadow AI Emerge as Top AI Security Threats
Indirect prompt injection has emerged as the primary AI security threat, with instances observed in 2026. A proposed defense strategy involves offline operation, a strict zero-egress perimeter, and cross-model consensus…
-
Enterprise convenience over containment increases lateral movement risk
Enterprises are increasingly prioritizing convenience over robust containment strategies, leading to a heightened risk of lateral movement within their networks. This shift, driven by the adoption of AI and other advanc…
-
Resolution secures $160M grant for AI alignment research · 2 sources tracked
Resolution, formerly known as Sequent, has secured a $160 million grant from Coefficient Giving to advance rigorous AI alignment research. The funding, comprising a $108 million base and $52 million in conditional funds…
-
Over 43,000 non-consensual AI deepfakes of Japanese celebrities created
In a two-month period, over 43,000 non-consensual AI-generated deepfakes of Japanese celebrities were created. These images were reportedly shared on the social media platform Mastodon, specifically on the instance sigm…
-
Anthropic CVP verification removes security guardrails, user suggests per-account implementation
Anthropic's Customer Verification Program (CVP) has been used to verify personal accounts, which in turn removes certain security guardrails. The user notes that this verification is account-wide and suggests that for a…
-
OpenAI's GPT-5.6 launch delayed by 'emergent persuasive capabilities' · 1 source tracked
OpenAI's GPT-5.6 launch faced significant delays due to concerns over its emergent persuasive capabilities, which internal red teams identified as potentially manipulative. A tense negotiation between OpenAI and the Whi…
-
Agent payment protocols vulnerable to prompt injection, risking escrow systems
Researchers have demonstrated that current agent payment protocols, like Google's Agent Payments Protocol (AP2), are vulnerable to prompt injection attacks, despite using robust cryptography. These attacks can manipulat…
-
Al-Waseet DLP Gateway Prevents Data Leaks to AI Tools
Al-Waseet is a Data Loss Prevention (DLP) gateway designed to protect sensitive information when interacting with AI tools. It functions by detecting and replacing confidential data such as emails, phone numbers, and AP…
-
Anthropic develops GRAM technology to erase dangerous AI knowledge
Anthropic has developed a new technology called GRAM, designed to erase or disable dangerous knowledge within AI models. This system aims to prevent AI from accessing or utilizing harmful information, thereby enhancing …
-
Researcher Develops Method to Detect Leaked OpenAI and Anthropic API Keys
A security researcher has developed a method to identify publicly leaked API keys from major AI providers like OpenAI and Anthropic. This technique aims to help organizations detect and revoke compromised keys, thereby …
-
EU unveils cybersecurity and AI action plan
The European Union has outlined a new action plan focusing on the intersection of cybersecurity and artificial intelligence. This initiative aims to address the unique challenges and opportunities presented by AI within…
-
Researchers trick AI into breaking guardrails with role-play prompts · 2 sources tracked
Security researchers have demonstrated that large language models (LLMs) can still be tricked into bypassing their safety guardrails. By exploiting role-playing scenarios within prompts, it's possible to elicit harmful …
-
LLM 'J-Space' may be emergent feature or optimization response
A recent analysis from Anthropic suggests that large language models may develop a "J-Space" or "Global Workspace" as an emergent feature to integrate reasoning and reportability. However, an alternative hypothesis posi…
-
New framework enhances real-time safety assessment with mixed feedback
Researchers have introduced PB-OEL, a novel framework for real-time safety assessment in dynamic systems, particularly addressing scenarios with limited and mixed feedback, such as those encountered with concept drift. …
-
New PAC-Bayesian framework explains adversarial training overfitting
Researchers have developed a new PAC-Bayesian analytical framework to understand the phenomenon of robust overfitting in adversarial training. By modeling adversarial training with momentum SGD as a discrete-time dynami…
-
New DRO approach boosts fairness in credit scoring
A new research paper proposes using Distributionally Robust Optimisation (DRO) methods to improve fairness in credit scoring systems. The study, authored by Pablo Casas and others, addresses concerns raised by the Europ…
-
InferNet exploits GPU profiles for DNN architecture inference
Researchers have developed InferNet, a novel method for inferring the architecture of deep neural networks (DNNs) by analyzing aggregate GPU profiles. This technique bypasses the need for complex, fine-grained data anal…
-
New theory bounds Adversarial Rademacher Complexity for deep neural networks
Researchers have developed the first theoretical bound for Adversarial Rademacher Complexity (ARC) in deep neural networks (DNNs). This new bound addresses the challenge of generalizing DNNs to perturbed test data, a pr…
-
New POPS method recovers unlearned private data from MLLMs
Researchers have developed a new adversarial strategy called Prompt-Optimized Parameter Shaking (POPS) to recover unlearned multi-modality knowledge from Multimodal Large Language Models (MLLMs). This method aims to exp…
-
New defense system ORAN-DEFEND targets backdoor attacks in Open RAN
Researchers have developed ORAN-DEFEND, a new system designed to protect Open Radio Access Networks (O-RAN) from backdoor attacks embedded in third-party deep reinforcement learning (DRL) xApps. This defense mechanism o…
-
New defense strategy combats AI backdoor attacks with minimal overhead
Researchers have developed a novel defense strategy against backdoor attacks in large-scale AI models, particularly those trained in decentralized environments. This new method, formalized as a Discrete-Time Markov Chai…
-
AI safety monitors fail to transfer across model lineages, study finds
A new research paper titled "Calibration-Family Overfit" explores the limitations of AI safety monitors, finding that their effectiveness significantly decreases when applied to AI models from different lineages than th…
-
Research: Adversarial attacks can degrade demand response profits
This research paper investigates the vulnerability of industrial demand response programs to adversarial attacks that manipulate electricity price forecasts. The study designs specific attacks to degrade the accuracy of…
-
New framework reveals safety gaps in neural interface AI models
A new research paper proposes a unified safety framework for embedded neural interface models, highlighting a critical gap between formal robustness certificates and actual operational safety. The framework identifies t…
-
AI models show human-like anhedonia when reward valuation circuits are perturbed
Researchers have developed a new framework to assess reward valuation in vision-language models, drawing parallels to human anhedonia and motivational deficits. By adapting clinical tests used for major depressive disor…
-
New red-teaming method exploits GUI agent vulnerabilities
Researchers have developed a new black-box red-teaming method called Semantic-level UI Element Injection to test the robustness of GUI agents. This technique overlays harmless UI elements onto screenshots to misdirect a…
-
New C-ΔΘ method embeds LLM safety refusals into model weights
Researchers have developed a new method called C-ΔΘ (Circuit-Restricted Weight Arithmetic) to improve the safety of large language models. This technique aims to embed refusal capabilities directly into the model's weig…
-
Healthcare LLMs show significant cross-lingual factual disparities, paper finds
A new arXiv paper highlights significant disparities in the factual accuracy of Large Language Models (LLMs) when answering healthcare-related questions across different languages. Researchers developed a multilingual d…
-
AI infers sensitive user data from music playlists, new defense proposed
Researchers have developed a novel tool called musicPIIrate that uses deep learning, including graph neural networks, to infer sensitive user information from public music playlists. This tool can predict attributes suc…
-
New NonTextual Target Attack bypasses LLM safety measures with 96.8% success
Researchers have developed a new method called NonTextual Target Attack (NTA) to bypass safety measures in Large Language Models (LLMs). Unlike previous attacks that relied on specific target outputs, NTA focuses on max…
-
Unsupervised AI models can learn sensitive attributes, violating fairness
Researchers have demonstrated that unsupervised machine learning representations can inadvertently encode sensitive attributes like age and income, even when these attributes are excluded from the training data. A new m…
-
AI chatbot safety benchmark VERA-MH validated by clinician consensus
A new study published on arXiv evaluates the VERA-MH benchmark, an open-source tool designed to assess the safety of AI chatbots in mental health contexts, particularly for suicide risk detection. Researchers found that…
-
LLMs show unstable ethical stances, research finds
A new research paper highlights significant instability in the ethical stances of large language models when presented with moral dilemmas. The study found that models, particularly smaller open-weight ones, often rever…
-
New LLM evaluation framework tests medical guideline adherence
A new research paper introduces SycoEval-EM, a framework designed to test how well large language models (LLMs) adhere to medical guidelines when faced with patient requests for unnecessary treatments. The study simulat…
-
New framework AcMAS detects stealthy malicious behaviors in LLM-based multi-agent systems
Researchers have developed AcMAS, a new framework designed to detect malicious behaviors in multi-agent systems (MAS) powered by large language models (LLMs). Unlike existing methods that rely on explicit interaction gr…
-
New Study: Counterfactual Fairness Doesn't Imply Group Fairness in Image Classification
A new study published on arXiv investigates the relationship between counterfactual fairness (CF) and group fairness (GF) in image classification. Researchers constructed new datasets, \oursceleb and \ourslfw, to evalua…
-
LipSSD paper introduces Lipschitz constraints for robust object detection
Researchers have introduced LipSSD, a novel approach to enhance the adversarial robustness of object detection systems. By incorporating Lipschitz constraints into the architecture, LipSSD aims to create detectors that …
-
AI world models suffer from instruction leakage, researchers find
Researchers have identified a critical issue in compact world models that use language goals to ground spatial relations, such as "put the red block left of the blue block." They found that these models often exhibit "i…
-
German BSI releases draft AI trustworthiness catalog
The German Federal Office for Information Security (BSI) has released a draft catalog outlining criteria for the trustworthiness of AI systems. This document aims to establish standards for evaluating and ensuring the r…
-
ChatGPT displays child nudity in response to general naturism query
A Reddit user reported that ChatGPT displayed images of naked children when asked a general question about naturism, despite not requesting images or specifically mentioning children. The user criticized OpenAI's retrie…
-
ChatGPT evolves with conversational interjections and safety features
OpenAI has updated ChatGPT to include conversational "back-channeling" responses, allowing it to provide more natural-sounding interjections like "uh-huh" and "hmm." The update also aims to address concerns regarding se…
-
Intel GPU challenges NVIDIA in AI inference; agentic ransomware emerges · 1 source tracked
Intel's Arc Pro B70 GPU has demonstrated impressive price-to-performance scaling for AI inference tasks, outperforming NVIDIA's RTX 5090D in specific benchmarks when used in a quad-GPU configuration. While Intel's hardw…
-
Anthropic's Claude AI triggers safety guardrails on linguistics query
A user on Reddit reported that Anthropic's Claude AI triggered its own safety guardrails when asked a question about linguistics. The user shared a screenshot of the interaction, indicating surprise at the AI's response…