Security scanners for AI agent skills agree no better than chance: Applying 5 security scanners to 3,084 skills revealed that the scanners disagreed on safety judgments in 64% of cases. Each scanner had different security
A study of five AI agent skill security scanners found that they agree on safety assessments less than 36% of the time. These scanners, which evaluate different security aspects like code vulnerabilities and prompt injection, frequently contradicted each other, with one scanner deeming a skill safe while another flagged it as critically dangerous in 14.2% of cases. This significant disagreement undermines the reliability of "safety" badges on skill marketplaces and highlights fundamental challenges in verifying the security of AI agent skills. AI
IMPACT Highlights significant challenges in trusting safety certifications for AI agent skills, potentially slowing adoption.