New attack collapses AI speculative decoding, increasing inference time

Researchers have developed a novel prompt-suffix attack called ADSD that exploits vulnerabilities in speculative decoding, a technique used to accelerate AI model inference. ADSD systematically pushes draft model probabilities towards tokens that the target model is unlikely to accept, causing a collapse in verifier acceptance. This attack can significantly increase inference time by up to 62.3% on datasets like GSM8K, while still preserving the quality of the task output. The researchers demonstrated that this vulnerability is present across various domains, speculative decoding strategies, and model architectures. AI

IMPACT This research highlights a critical security vulnerability in AI inference acceleration, potentially impacting the efficiency and reliability of deployed models.

RANK_REASON Academic paper detailing a new attack method on AI inference techniques. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• GSM8K
• Soft-Collapse
• speculative decoding

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →