LRU
PulseAugur coverage of LRU — every cluster mentioning LRU across labs, papers, and developer communities, ranked by signal.
4 day(s) with sentiment data
-
AI agents to report tool progress for better performance
Researchers have developed a new method for AI agents to better manage their progress when using external tools. Current systems often guess how long a tool will take, leading to inefficient use of GPU memory. The propo…
-
LRU Caching Proves Surprisingly Resilient Against Advanced AI Cache Methods
A recent article suggests that Least Recently Used (LRU) caching strategies are more effective than previously understood, even when compared to advanced KV-cache techniques used in AI models. The author highlights that…
-
New H-MC caching policy offers improved regret guarantees over LRU/LFU
Researchers have developed a new caching policy called H-MC that aims to improve upon existing methods like LRU and LFU. Unlike previous policies such as LeCar and Cacheus, which can suffer from linear regret against ce…
-
Developers can fix LLM API 429 errors with caching
Developers encountering frequent HTTP 429 "Too Many Requests" errors from free-tier LLM APIs can often resolve the issue by implementing a cache. These errors, especially when occurring during peak hours, may stem from …
-
LLM Cache Study: LFU Outperforms Other Eviction Policies, But Effectiveness Limited
A new study published on arXiv evaluates various eviction policies for Large Language Model (LLM) caches, finding that the Least Frequently Used (LFU) policy performs best among those tested. The research, conducted usi…
-
MoE expert caching evaluation methods found to be misleading
Researchers have identified critical flaws in trace-driven evaluation methods for Mixture-of-Experts (MoE) models, which can lead to misleading conclusions about expert caching policies. The study highlights how replay …
-
New analysis explores caching strategies for Genetic Programming Symbolic Regression
This paper analyzes caching strategies for Genetic Programming Symbolic Regression (GPSR) to reduce computational runtime. The study explores memory-runtime trade-offs of various caching mechanisms, including LRU and FI…
-
LLM KV Cache Management Needs Rethink Beyond Simple Eviction
The KV cache in LLM inference servers is often managed with a simple LRU or capacity-based eviction policy, which is an inadequate approach for modern workloads. This strategy fails to account for the varying future val…
-
Phionyx introduces deterministic AI runtime for enhanced governance
Researchers have introduced Phionyx, a novel AI runtime architecture designed for deterministic behavior and enhanced governance. This architecture treats LLM outputs as noisy measurements, employing deterministic state…
-
KV cache eviction protection proves more vital than scoring
Researchers have developed a new method for managing KV cache eviction in large language models, finding that structural protection is more critical than scoring algorithms. Their study on transformer models revealed th…
-
Looped SSMs improve time series classification with depth-recurrence
Researchers have introduced Looped SSMs, a novel approach to State Space Models for time series classification. This method enhances performance by applying depth-recurrence, where model blocks are reused across layers,…
-
Apple researchers unveil SpecMD for faster MoE model inference
Apple's machine learning research team has published a paper detailing SpecMD, a new framework for evaluating Mixture-of-Experts (MoE) model caching policies. Their experiments show that traditional caching assumptions …
-
New ML-based GPU caching algorithm LCR boosts LLM inference speed
Researchers have developed a new GPU caching algorithm called Learning-Augmented LRU (LALRU) designed to improve efficiency during AI inference. This algorithm integrates learned predictions with caching policies to ens…
-
Memristor-based AI systems show promise for efficient learning and neuromorphic computing
Researchers are exploring Self-Organising Memristive Networks (SOMNs) as a physical alternative to conventional hardware for artificial intelligence, aiming for energy-efficient, brain-like continual learning. These net…