DLRM

New ML-based GPU caching algorithm LCR boosts LLM inference speed

Researchers have developed a new GPU caching algorithm called Learning-Augmented LRU (LALRU) designed to improve efficiency during AI inference. This algorithm integrates learned predictions with caching policies to ens…