Researchers have developed MaliciousSkillBench, a new benchmark designed to detect malicious agent skills, which are extensions that can provide LLM agents with scripts and resources. The benchmark consolidates data from 13 public sources, resulting in a dataset of 9,740 skills, with 7,505 identified as malicious and 2,235 as benign. Evaluations of various detection methods, including learned text detectors and off-the-shelf scanners, revealed that while some models achieve high recall for malicious skills, they often struggle with a high false positive rate on benign skills, indicating a need for more robust detection strategies. AI
IMPACT This benchmark could lead to more reliable detection of malicious code embedded in AI agent extensions, enhancing overall AI security.
RANK_REASON The item is a research paper introducing a new benchmark for detecting malicious AI agent skills. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
