David Grangier

Apple researchers propose cache sharing to reduce LLM serving costs

Apple Machine Learning Research has published a paper detailing a new method called Stochastic KV Routing to reduce the memory footprint of transformer language models. This technique focuses on optimizing the depth dim…