PulseAugur
EN
LIVE 15:47:44

llama.cpp releases b12170 with new moe-cache hybrid system

By PulseAugur Editorial · [1 sources] ·

The llama.cpp project has released version b12170, introducing a new hybrid cache mechanism called moe-cache. This cache utilizes a combination of LRFU (Least Recently Used First) and MQ (Multi-Queue) principles, along with a ClassLRU implementation. The update includes optimizations for cache key computation and scoring, and has been tested across various operating systems including macOS, iOS, Linux, Android, and Windows, with specific configurations for Apple Silicon and openEuler. AI

IMPACT Improves performance and efficiency for local LLM inference on various devices.

RANK_REASON This is a software release for a specific project, not a frontier model release or significant industry event.

Read on llama.cpp — Releases →

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →

llama.cpp releases b12170 with new moe-cache hybrid system

How we ranked this

Signal score
6 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
This is a software release for a specific project, not a frontier model release or significant industry event.
Source corroboration
Single-source cluster
Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics
infra
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
Same-day
Cluster formed today. Ranking reflects the current source set at time of score.

Full methodology in our editorial standards.

COVERAGE [1]

  1. llama.cpp — Releases TIER_1 English(EN) · sanmai ·

    b12170: moe-cache : LRFU/MQ-like hybrid cache, ClassLRU (#31233)

    <ul> <li> <p>moe-cache : LRFU/MQ-inspired hybrid cache</p> </li> <li> <p>make the clock advance by N tokens: use the real token count (ne[1] is always 1, ids have n_expert_used entries per token)</p> </li> <li> <p>sel is a RESHAPE node, the orig input is in src[0], so sel-&gt;src…

RELATED ENTITIES

RELATED TOPICS