Rowhammer attack amplifies LLM inference latency and cost

Researchers have developed LLVMhammer, a novel Rowhammer attack that targets the latency and cost of Large Language Model (LLM) inference on GPUs. By inducing a bit-flip in the CPU's DRAM, the attack causes the Triton JIT compilation framework to generate suboptimal GPU kernels, leading to increased inference latency. This slowdown, ranging from 4-16%, can translate to higher operational costs for multi-GPU deployments. The attack is GPU-agnostic and can also corrupt LLM outputs, impacting metrics like BERTScore. AI

IMPACT Highlights potential security vulnerabilities in LLM inference pipelines, impacting cost and reliability.

RANK_REASON Research paper detailing a novel security vulnerability in LLM inference. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• AMD RX 7900 XTX
• BERTScore
• Deepseek-llm-7B-chat
• DRAM
• GPU
• Llama 3.1 8B-Instruct
• LLM
• LLVMhammer
• Mistral 7B Instruct v0.3
• NVIDIA RTX 4090
• Qwen2.5-7B-Instruct
• Triton

• safety
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →