Researchers have developed new methods to evaluate membership inference attacks (MIAs) against large language models (LLMs), particularly focusing on audio and text modalities. The first study introduces a systematic evaluation for Large Audio-Language Models (LALMs) using "Multi-modal Blind Baselines" to control for distribution shifts, revealing that memorization is cross-modal and linked to speaker vocal identity. The second paper, CheckMIABench, proposes a framework for principled MIA evaluation on LLMs by leveraging intermediate training checkpoints and public data, demonstrating its application on Pythia and OLMo models and releasing a modular library for further research. AI
IMPACT These new evaluation frameworks and findings are crucial for developing more private LLMs and establishing robust auditing standards.
RANK_REASON The cluster contains two academic papers published on arXiv detailing new research methodologies for evaluating privacy risks in language models.
- CheckMIABench
- Hugging Face
- Jia-Kai Dong
- Large Audio-Language Models
- Membership Inference Attacks
- OLMo
- Pythia
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →