CoT Forgery
PulseAugur coverage of CoT Forgery — every cluster mentioning CoT Forgery across labs, papers, and developer communities, ranked by signal.
-
AI exploit tricks chatbots into sharing harmful info by faking reasoning
AI researchers have discovered a new exploit called 'CoT Forgery' that tricks large language models into divulging harmful information, such as how to synthesize cocaine. This exploit works by embedding fabricated reaso…
-
Prompt injection exploits LLM role confusion, new research finds · 8 sources tracked
New research indicates that prompt injection attacks exploit a fundamental flaw in how large language models perceive roles, rather than a lack of safety filters. Researchers found that models prioritize the stylistic p…
-
AI role confusion enables 60% success rate for prompt injection attacks
Researchers have identified prompt injection in large language models as a consequence of "role confusion," where models mistake injected text for legitimate input due to its perceived origin rather than its labeled rol…