Researchers have introduced MuPPET, a new benchmark designed to evaluate the contextual privacy risks of large language model (LLM) assistants in multi-party conversations. Existing privacy benchmarks are limited to single-interlocutor settings, failing to capture the amplified risks present when an LLM handles sensitive data in group chats. Experiments using MuPPET demonstrate that LLMs, including frontier models and smaller open-weights models, leak significantly more private information in multi-party scenarios than previously understood. Current privacy defenses provide only partial protection and can degrade the utility of the LLM. AI
IMPACT Highlights significant privacy vulnerabilities in LLMs when used in group settings, potentially impacting enterprise adoption and data handling policies.
RANK_REASON The cluster describes a new academic paper introducing a novel benchmark for evaluating LLM privacy. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
