Membership Inference Attacks against Large Audio Language Models
Researchers have developed new methods to evaluate membership inference attacks (MIAs) against large language models (LLMs), particularly focusing on audio and text modalities. The first study introduces a systematic evaluation for Large Audio-Language Models (LALMs) using "Multi-modal Blind Baselines" to control for distribution shifts, revealing that memorization is cross-modal and linked to speaker vocal identity. The second paper, CheckMIABench, proposes a framework for principled MIA evaluation on LLMs by leveraging intermediate training checkpoints and public data, demonstrating its application on Pythia and OLMo models and releasing a modular library for further research. AI
IMPACT These new evaluation frameworks and findings are crucial for developing more private LLMs and establishing robust auditing standards.