On-device LLM safety analysis reveals sparse critical parameters

Researchers have investigated the safety of on-device language models, specifically Llama-2-7B-Chat, to determine if safety-critical parameters are concentrated in sparse subsets. Their analysis revealed that safety sensitivity is unevenly distributed across the model, with the MLP down_proj consistently showing high sensitivity. By modifying a small fraction of weights (0.19%) in the down_proj layer, they achieved significant increases in adversarial success rates (ASR) while maintaining baseline accuracy on tinyBenchmarks, suggesting a targeted approach for analyzing and protecting on-device models. AI

IMPACT Suggests methods for targeted analysis and protection of on-device LLMs, potentially improving security for edge AI applications.

RANK_REASON The cluster contains an academic paper detailing research findings on LLM safety. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• arXiv
• cs.CL
• down_proj
• Llama-2-7B-Chat
• Muhammad Zeeshan Karamat
• multilayer perceptron

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →