arXiv:2610.12327v1 Announce Type: cross Abstract: The memory-bound nature of the decoding stage of large language model (LLM) inference incurs significant latency. Layer-wise training-free network pruning approaches guided by the Hessian have been a prominent solution to this pro…
arXiv cs.CL
TIER_1English(EN)·Jorge L. Ruiz Williams·
arXiv:2602.06317v3 Announce Type: replace-cross Abstract: Attention-mass concentration creates an opportunity for sparse decoding, but retained mass alone does not guarantee a stable greedy decision: retrieval error, omitted value directions, and recursive decoding all matter. We…
The memory-bound nature of the decoding stage of large language model (LLM) inference incurs significant latency. Layer-wise training-free network pruning approaches guided by the Hessian have been a prominent solution to this problem, as pruning reduces the number of nonzero par…
arXiv cs.AI
TIER_1English(EN)·Noam Elata, Itay Lamprecht, Mikey Shechter, Daniel Ohayon, Itay Hubara, Daniel Soudry·
arXiv:2610.04753v2 Announce Type: replace-cross Abstract: Autoregressive generation in Large Language Models (LLMs) is constrained by the memory and computational demands of attention mechanisms. Sparse attention methods mitigate this cost by selecting only high-probability entri…
arXiv cs.LG
TIER_1English(EN)·JuneHyung Kim, Sankeerth Durvasula, Nandita Vijaykumar·
arXiv:2610.02598v1 Announce Type: new Abstract: Deploying LLMs on consumer-grade GPUs with insufficient memory to hold their weights can result in prohibitively slow inference, because decoding repeatedly transfers offloaded weights from system RAM or flash storage into GPU at mu…