Researchers have developed a new method called JEPA-Prune to address privacy concerns in large language models (LLMs) after fine-tuning. This technique aims to remove personally identifiable information (PII) without significantly degrading the model's performance. JEPA-Prune uses a joint-embedding predictive architecture (JEPA) to train a reference model on PII-free data, allowing it to identify and prune PII-specific neurons in the fine-tuned model. Experiments indicate that pruning a small percentage of neurons can drastically reduce PII inference accuracy and extraction recall while maintaining utility. AI
IMPACT This research offers a novel approach to mitigate privacy risks in LLMs, potentially enabling safer deployment of fine-tuned models on sensitive data.
RANK_REASON The cluster contains an academic paper detailing a new method for LLM privacy. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
