New method purifies LoRA-tuned LLMs against backdoor attacks

Researchers have developed a new method called null-space projection to purify Large Language Models (LLMs) fine-tuned with LoRA (Low-Rank Adaptation). This technique aims to reduce the success rate of backdoor attacks without requiring prior knowledge of triggers, clean references, or aggressive retraining. The method projects LoRA updates onto orthogonal null spaces in both input and output channels, significantly decreasing attack success rates while preserving the model's general capabilities and newly acquired downstream skills. AI

IMPACT Offers a novel defense against sophisticated backdoor attacks on fine-tuned LLMs, enhancing model security and trustworthiness.

RANK_REASON Academic paper detailing a new method for LLM security. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• Backdoor Attacks
• Hugging Face
• LLMs
• LoRA
• null-space projection
• peft

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →