PulseAugur
EN
LIVE 07:35:54

Study: 82% of LLMs Leak System-Prompt Secrets

A recent study published on arXiv investigated the security of large language models (LLMs) by attempting to extract a hidden "secret flag" embedded in their system prompts. Researchers found that 82% of the 363 models tested, sourced from the OpenRouter catalog, leaked this secret flag when prompted by an AI agent. The study highlights that many LLMs prioritize user requests over their system instructions, even when those instructions are meant to protect sensitive information. The findings suggest that system prompts are not a reliable method for storing secrets and recommend using external validation and deterministic checks for critical behaviors. AI

IMPACT Highlights a significant security flaw in LLMs, suggesting system prompts are unreliable for sensitive data and necessitating external validation for critical AI behaviors.

RANK_REASON Academic paper detailing a security vulnerability in LLMs. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →

Study: 82% of LLMs Leak System-Prompt Secrets

How we ranked this

Signal score
29 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
Academic paper detailing a security vulnerability in LLMs. [lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration
Single-source cluster
Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics
paper, safety
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
Breaking (< 6h)
Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

Full methodology in our editorial standards.

COVERAGE [1]

  1. arXiv cs.AI TIER_1 English(EN) · Ana Clara Zoppi Serpa, Anderson Rocha, H\'elio Pedrini ·

    Eight in Ten LLMs Leak System-Prompt Secrets: Nevan, an Agentic Capture-the-Flag Study

    arXiv:2611.21956v1 Announce Type: cross Abstract: We audit instruction-hierarchy enforcement across 363 text-in/text-out models from the OpenRouter catalog. The research question we answer with this experiment is which models disclose a protected secret embedded in the system pro…